In Part 2 — Installing Jenkins Controller on AWS EC2, we installed Jenkins on a single EC2 instance and intentionally allowed the built-in node to execute our first Pipeline. That gave us the simplest possible Jenkins environment, but it also exposed the main weakness of the all-in-one model: orchestration and build execution share the same machine.
Part 3 separates those responsibilities.
The existing EC2 instance remains the Jenkins Controller, while a second EC2 instance becomes a static Jenkins Agent. The Controller keeps responsibility for the Queue, scheduling, Pipeline orchestration, configuration, and Jenkins state. The Agent provides the Executor, Workspace, build tools, and compute resources used by the Pipeline.
By the end of Part 3, our architecture will move from Controller + build workload on one EC2 instance to a traditional distributed Jenkins model with a dedicated Controller and a dedicated build worker.
1. Why Move Builds Away from the Controller
The all-in-one environment from Part 2 was useful precisely because its limitation was easy to see.
A build running on the built-in node consumes the same CPU, memory, disk I/O, and operating-system resources that Jenkins needs for its own orchestration work. A badly behaved build can therefore affect the Controller itself.
Separating execution changes that boundary:
The goal is not simply to add another server. The goal is to make control and execution independently manageable.
That gives us a cleaner failure boundary, dedicated build resources, workload-specific toolchains, and a path toward increasing execution capacity without increasing the size of the Controller itself.
2. Architecture and EC2 Agent Baseline
We will reuse the Jenkins Controller from Part 2 and add one static EC2 Agent. For this learning lab, the Agent runs in the same VPC as the Controller, in a public subnet with a public IPv4 address. Engineers still administer it through AWS Systems Manager Session Manager, while Jenkins reaches it over the Agent's private IP or internal DNS name using SSH from the Controller Security Group.
Because the Agent has a public IPv4 address and runs in a public subnet, this lab does not need a NAT Gateway. The subnet route table sends 0.0.0.0/0 directly to the VPC Internet Gateway.
That direct outbound path allows the SSM Agent to establish HTTPS connections to AWS Systems Manager and lets the Ubuntu host reach package repositories, Git services, and other build dependencies used later in the lab.
Session Manager does not require inbound TCP/443 to the EC2 instance. The SSM Agent initiates the connection outbound. The Agent Security Group therefore only needs the Jenkins-specific inbound rule used in Part 3: TCP/22 from the Controller Security Group. The public IPv4 is not used by Jenkins for SSH.
EC2 infrastructure baseline
For this lab, use the following EC2 configuration:
| EC2 parameter | Baseline | Why |
|---|---|---|
| Name | jenkins-agent-01 | Clear separation from the Controller instance |
| AMI / OS | Ubuntu Server 24.04 LTS | Consistent with Part 2 |
| Instance type | t3.small | Sufficient for the lightweight Pipeline used in this lab |
| Root volume | 20 GiB gp3 | Enough for Java, Git, Jenkins Workspace, and small build outputs |
| VPC | Same VPC as the Jenkins Controller | Allows private Controller-to-Agent connectivity |
| Subnet | Public subnet | Keeps the lab simple and provides direct outbound Internet connectivity through the Internet Gateway |
| Public IPv4 | Enabled | Provides outbound Internet access without requiring a NAT Gateway; Jenkins still uses the Agent private IP |
| Inbound Security Group | TCP/22 from the Controller Security Group only | Allows Jenkins to launch the Agent over SSH without exposing SSH publicly |
| Administrative access | AWS Systems Manager Session Manager | No public SSH endpoint is required for engineers |
| IAM instance profile | AmazonSSMManagedInstanceCore baseline | Allows the instance to register with Systems Manager and be administered through Session Manager |
| Public subnet route | 0.0.0.0/0 → Internet Gateway | Provides direct outbound connectivity for Systems Manager, package installation, Git, registries, and build dependencies |
| NAT Gateway | Not required for this lab | The Agent has a public IPv4 address and uses the public subnet's Internet Gateway route for egress |
| Security Group egress | Use the lab's default outbound policy, or explicitly allow the destinations and ports required by SSM, package repositories, Git, registries, and build dependencies | SSM Agent initiates outbound HTTPS connections; package and build traffic may require additional outbound ports depending on the repositories and tools in use |
The t3.small instance is only a lab baseline. Agent sizing is workload-dependent: Java compilation, container builds, Terraform, security scanners, or parallel tests may require very different CPU, memory, and disk characteristics.
Together, these settings give us the intended lab boundary: public subnet + public IPv4 for outbound access, no public SSH, Session Manager for engineers, and private TCP/22 connectivity from the Jenkins Controller to the Agent.
ℹ️ This is a learning architecture, not the production target. In production, Jenkins Agents are typically better placed in private subnets with controlled outbound connectivity through NAT Gateway, VPC endpoints, proxies, or another approved egress design. We use a public IPv4 here to keep Part 3 focused on the Jenkins Controller–Agent model rather than on additional VPC egress infrastructure.
Jenkins Agent baseline
Once the EC2 instance exists, Jenkins turns that compute into a schedulable execution node using the following configuration:
| Jenkins setting | Value |
|---|---|
| Node name | ec2-agent-01 |
| Java runtime | OpenJDK 21 |
| Executors | 1 |
| Remote root directory | /home/jenkins/agent |
| Labels | ec2 linux static |
| Launch method | Launch agents via SSH |
| SSH user | jenkins |
| Host | Agent private IP or internal DNS name |
This distinction is important:
EC2 defines the compute and network boundary. Jenkins node configuration turns that compute into schedulable execution capacity.
The Agent does not need the Jenkins Controller application installed. It needs a supported Java runtime for Jenkins Remoting and whatever build tools the workload requires.
For our small Pipeline, Java, Git, and a shell are sufficient. Real Agents may additionally include Maven, Node.js, Docker-compatible tooling, Terraform, scanners, or deployment CLIs.
These values define the target Jenkins Agent baseline. The EC2 instance exists, but Jenkins does not know about it yet. In the sections that follow, we will prepare the runtime, establish SSH trust, create the Jenkins node, apply the labels and Executor settings shown above, and verify that ec2-agent-01 becomes Online.
🧭 We are deliberately keeping the Agent static. The EC2 instance exists before the build is requested and remains running after the build completes. That behavior will become the main architectural limitation we examine at the end of Part 3.
3. How the Controller Connects to an Agent
Jenkins supports multiple ways to connect Controllers and Agents. Two common models are SSH-launched Agents and inbound Agents.
| Model | Connection direction | Good fit |
|---|---|---|
| SSH Agent | Controller → Agent | Static Linux hosts reachable from the Controller |
| Inbound Agent | Agent → Controller | Environments where the Agent initiates connectivity or sits behind restrictive network boundaries |
With the SSH model used in Part 3, the Controller authenticates to the EC2 Agent using an SSH credential. The SSH Build Agents plugin starts the Jenkins remoting process on the Agent, after which the Controller and Agent communicate through the Jenkins remoting channel.
Conceptually:
For a static Linux EC2 instance reachable over the same VPC, this model is straightforward to inspect and troubleshoot, which makes it a good fit for this stage of the series.
Inbound Agents become more interesting once Agents are created dynamically or the network direction needs to be reversed. We will revisit Agent connectivity when we move toward ephemeral compute.
4. Prepare the Static EC2 Agent
With the EC2 Agent from Section 2 running, open a Systems Manager Session Manager session to it. The instance has a public IPv4 address for outbound connectivity, but engineers still do not SSH to that public address. There is no need to temporarily open TCP/22 from your own IP.
Before continuing, verify that the instance is visible as a managed node in Systems Manager and that the SSM Agent is running.
From the Session Manager shell, install Java 21, Git, and ensure the SSH service is available for the Jenkins Controller:
sudo apt update sudo apt install -y fontconfig openjdk-21-jre git openssh-server java --version sudo systemctl enable --now ssh sudo systemctl status ssh --no-pager Create a dedicated operating-system user and Agent directory:
sudo useradd --create-home --shell /bin/bash jenkins sudo -u jenkins mkdir -p /home/jenkins/agent Next, configure the Controller-to-Agent SSH credential.
We will create a dedicated SSH key pair on the Jenkins Controller, copy only the public key to the Agent, and later store the private key in Jenkins Credentials.
Generate the SSH key pair on the Jenkins Controller
Open a shell on the Jenkins Controller and create a dedicated key pair for this Agent:
sudo -u jenkins mkdir -p /var/lib/jenkins/.ssh sudo chmod 700 /var/lib/jenkins/.ssh sudo -u jenkins ssh-keygen \ -t rsa \ -b 4096 \ -f /var/lib/jenkins/.ssh/ec2-agent-01 \ -C "jenkins-controller-to-ec2-agent-01" \ -N '' This creates:
/var/lib/jenkins/.ssh/ec2-agent-01 # private key /var/lib/jenkins/.ssh/ec2-agent-01.pub # public key Display the public key:
sudo -u jenkins cat /var/lib/jenkins/.ssh/ec2-agent-01.pub Copy the complete output. It should look similar to:
ssh-rsa AAAA... jenkins-controller-to-ec2-agent-01 Do not copy the private key to the Agent.
Add the public key to the Jenkins Agent
Now open a Session Manager shell on jenkins-agent-01 and prepare the Jenkins user's SSH directory:
sudo -u jenkins mkdir -p /home/jenkins/.ssh sudo chmod 700 /home/jenkins/.ssh Add the public key generated on the Controller:
sudo -u jenkins tee -a /home/jenkins/.ssh/authorized_keys > /dev/null <<'EOF' <PASTE_THE_CONTROLLER_PUBLIC_KEY_HERE> EOF Then apply the expected permissions:
sudo chmod 600 /home/jenkins/.ssh/authorized_keys sudo chown -R jenkins:jenkins /home/jenkins/.ssh The trust relationship now looks like this:
In the next section, we will import the Controller's private key into Jenkins Credentials and use that credential for the SSH-launched Agent. Session Manager remains the human-administration channel, while the SSH key is reserved for Jenkins' private Controller-to-Agent connection.
Avoid reusing a personal administrative SSH key for Jenkins automation. The Controller only needs enough SSH access to launch and manage the Agent process as the dedicated jenkins user.
ℹ️ The Java version used by the Jenkins Controller and Agent processes must follow the Jenkins Java support policy. This series uses Java 21 for both. The JDK used by an application build can be different from the JVM that runs Jenkins itself.
5. Configure the Agent in Jenkins
Before creating the node, confirm that the SSH Build Agents plugin is installed. If it is not present, install it from Manage Jenkins → Plugins.
Next, add the SSH private key generated in Section 4 to Jenkins Credentials.
On the Jenkins Controller, display the private key:
sudo -u jenkins cat /var/lib/jenkins/.ssh/ec2-agent-01 Copy the complete private-key block, including the BEGIN and END lines.
In Jenkins, create a credential with:
- Kind: SSH Username with private key
- Username:
jenkins - Private Key: Enter directly, then paste the generated private key
- ID:
ec2-agent-ssh
After the credential has been saved successfully, remove the temporary key-pair files from the Controller filesystem:
sudo rm -f /var/lib/jenkins/.ssh/ec2-agent-01 sudo rm -f /var/lib/jenkins/.ssh/ec2-agent-01.pub Jenkins should now use the managed credential rather than a private-key file left on the host.
Then go to Manage Jenkins → Nodes → New Node and create a permanent Agent.
Use a configuration similar to:
| Setting | Value |
|---|---|
| Node name | ec2-agent-01 |
| Number of executors | 1 |
| Remote root directory | /home/jenkins/agent |
| Labels | ec2 linux static |
| Usage | Only build jobs with label expressions matching this node |
| Launch method | Launch agents via SSH |
| Host | Agent private IP or internal DNS name |
| Credentials | ec2-agent-ssh |
| Host Key Verification Strategy | Non verifying Verification Strategy |
For this learning lab, select Non verifying Verification Strategy. This avoids the additional known_hosts setup and keeps the lab focused on the Controller–Agent SSH flow.
⚠️ Production note: Non-verifying host-key validation disables verification of the Agent's SSH host identity and therefore removes protection against SSH man-in-the-middle attacks. In production, use Known hosts file Verification Strategy or another verification strategy that validates the Agent host key. With the known-hosts strategy, Jenkins expects the trusted host key in /var/lib/jenkins/.ssh/known_hosts on the Controller.
Once Jenkins connects successfully, the node should become Online.
On the Agent, you can confirm that Jenkins has started the Java/remoting process:
ps -ef | grep '[j]ava' The important transition is now visible: the Controller still owns orchestration, while the Agent now owns the execution environment.
6. Disable Build Execution on the Controller
Now that a dedicated Agent exists, the built-in node should stop providing build capacity.
Go to Manage Jenkins → Nodes → Built-In Node → Configure and set:
Number of executors = 0 This is more than a cosmetic change. With zero Executors on the built-in node, Jenkins can no longer silently fall back to running build workloads on the Controller. The Controller remains responsible for scheduling and orchestration, while build execution must occur on an Agent.
This is the first point in the series where the Controller–Agent boundary is enforced by configuration rather than merely described in a diagram.
7. Use Labels to Control Workload Placement
A Jenkins Agent becomes much more useful when we can express what kind of workload should run there.
Our Agent has the labels:
ec2 linux static Modify the Pipeline from Part 2 so that it explicitly requests the EC2 Agent:
pipeline { agent { label 'ec2' } stages { stage('Inspect Environment') { steps { sh ''' echo "NODE_NAME=$NODE_NAME" hostname pwd java -version ''' } } stage('Build') { steps { sh 'echo "building..."' } } stage('Test') { steps { sh 'echo "testing..."' } } stage('Package') { steps { sh 'echo "hello Jenkins Agent" > artifact.txt' archiveArtifacts artifacts: 'artifact.txt' } } } } The Pipeline now expresses a scheduling requirement instead of accepting any execution environment. The Controller evaluates the label expression, finds an online Agent with a matching label, waits for an available Executor, and schedules the workload there.
That is the practical meaning of workload placement in Jenkins.
💡Labels should describe useful workload capabilities or placement intent—such as linux, java, terraform, or gpu—rather than becoming arbitrary names that Pipelines cannot reason about.
8. Observe the Distributed Pipeline Lifecycle
Run the Pipeline again.
The logical lifecycle has not changed:
Trigger → Queue → Executor → Workspace → Build → Artifact
What changed is where execution happens.
The Controller receives and schedules the build, but the selected Executor now belongs to ec2-agent-01. The shell commands execute on the Agent, and the Workspace is created beneath the Agent's remote root directory.
Inspect it directly on the Agent:
sudo -u jenkins ls -lah /home/jenkins/agent/workspace/ You should see the job Workspace under a path similar to:
/home/jenkins/agent/workspace/hello-jenkins The hostname and pwd output in Console Output should also confirm that the build is no longer running on the Controller.
Conceptually:
When archiveArtifacts runs, the file originates in the Agent Workspace and is copied into Jenkins' build record. The Workspace remains local to the execution node, while the archived Artifact survives independently of that Workspace.
This is the same distinction introduced in Part 1 — Jenkins Fundamentals: What Are We Actually Operating?, but now the storage boundary spans two machines.
9. Give the Agent AWS Access Without Static Credentials
The EC2 Agent already has an IAM instance profile in this lab because Session Manager needs Systems Manager permissions. Treat that as the administration baseline, not as permission for application builds.
A clean model is:
- Baseline role permissions: Systems Manager access required to administer the EC2 Agent.
- Workload permissions: additional least-privilege permissions only when the Jenkins build actually needs AWS APIs.
Do not solve AWS access by storing long-lived access keys on the server or hard-coding them into Jenkins.
For an EC2 Agent, the natural AWS credential boundary is the Agent's IAM instance profile. The AWS SDK and CLI can obtain temporary credentials from the EC2 metadata service, while the IAM role defines what that execution environment is allowed to do.
The Controller does not need to inherit those permissions.
One important limitation is that every build running on the same EC2 Agent shares the same instance-profile identity. If Java builds, Terraform, and deployment workloads require materially different AWS permissions, place them on separate Agent classes or give the Agent a narrow base role that can assume only the approved workload-specific roles.
For example, a build Agent might need read access to ECR, while a deployment Agent may be allowed to assume a specific deployment role. Those permissions should follow the execution boundary rather than being granted globally to the Jenkins Controller.
If the AWS CLI is installed, you can verify the Agent identity with:
aws sts get-caller-identity The important design principle is:
Human administration uses Session Manager, Jenkins connectivity uses private SSH, and AWS API permissions follow the Agent workload through temporary IAM credentials.
10. Failure Signals Worth Understanding
A distributed Jenkins setup introduces a few new failure modes. We do not need a full troubleshooting guide yet, but these signals are worth recognizing.
| Symptom | What to check first |
|---|---|
| Agent is not visible in Session Manager | SSM Agent, IAM instance profile, outbound path to Systems Manager |
| Controller cannot SSH to Agent | Private IP/DNS, Agent Security Group allowing TCP/22 from Controller SG, NACLs, and sshd |
| Agent is offline in Jenkins | SSH credential, host-key verification, Java runtime, remote root permissions |
| Build waits in Queue | Label expression, Agent online state, available Executor |
| SSH connects but Agent does not start | Java availability, remote root permissions, Jenkins launch log |
| Build starts but cannot write files | Workspace ownership and filesystem permissions |
A particularly useful Queue message is a label mismatch. If a Pipeline requests a label for which no online Agent is available, Jenkins can accept the build request but cannot schedule execution.
That is a concrete example of the distinction introduced in Part 1: triggering a build is not the same as having execution capacity for that build.
11. What Static EC2 Agents Solve
The move from the built-in node to a dedicated Agent gives us several immediate improvements.
Controller isolation. Build CPU, memory, disk I/O, and temporary files are no longer competing directly with Jenkins orchestration on the same host.
Workload-specific environments. Agents can carry different tools and labels. A Java Agent does not need the same software stack as a Terraform or Node.js Agent.
Independent permissions. Different Agent classes can use different IAM instance profiles or approved AssumeRole paths instead of attaching broad workload permissions to the Controller.
Clearer failure boundaries. A failed build worker does not automatically mean the Jenkins Controller itself has failed.
Independent capacity. We can add more Agents without increasing the size of the Controller.
Using one Executor for this first Agent gives us a predictable baseline: one Agent, one execution slot, and one active Workspace workload at a time. Higher Executor counts may improve utilization for lightweight jobs, but they also make concurrent builds compete for the same CPU, memory, disk, and IAM identity.
This architecture is already much healthier than running builds directly on the Controller.
But it introduces another problem: the EC2 Agent exists whether Jenkins needs it or not.
12. What Static Agents Do Not Solve
Our Agent is now isolated, but it is still static capacity.
The instance is provisioned before demand exists. It remains powered on while no build is running, and its CPU/memory shape is fixed until someone changes it.
As the number of workloads grows, several operational problems appear:
- Idle cost: EC2 capacity continues to exist when the Queue is empty.
- Manual provisioning: new capacity requires another instance, runtime setup, credentials, node configuration, and validation.
- Configuration drift: long-running Agents accumulate package changes, tool versions, temporary files, and local differences.
- Fixed sizing: a small Agent may be insufficient for a heavy job, while a large Agent may be wasteful for small jobs.
- Workspace cleanup: persistent workers accumulate workspaces, caches, and temporary files unless cleanup is managed explicitly.
- Slow scaling: adding capacity is an operational action rather than an automatic response to Queue demand.
- Host lifecycle: patching, Java upgrades, disk pressure, and operating-system maintenance now apply to every Agent host.
The architecture has solved where builds should run, but not how build capacity should appear and disappear with demand.
That distinction leads directly to Part 4.
13. Verify What You Built
Before moving on, make sure the environment demonstrates the architectural change rather than merely containing a second EC2 instance.
- [ ]
ec2-agent-01is Online in Jenkins. - [ ] The Agent connects using a dedicated SSH credential over its private IP or internal DNS name.
- [ ] The built-in node has
0Executors. - [ ] The Pipeline requests the
ec2label. - [ ] Console Output shows the Agent hostname and Agent Workspace path.
- [ ]
artifact.txtis created in the Agent Workspace and appears as an archived Artifact in Jenkins. - [ ] AWS permissions, when needed, come from the Agent IAM role rather than static access keys.
- [ ] You can explain what happens if the Agent is offline or no matching Executor is available.
If all of these are true, the Controller–Agent separation is now real.
14. Cost and Cleanup
The Jenkins Controller from Part 2 and the static Agent both incur EC2/EBS cost while they exist. The important difference is that the Agent may incur cost even when there are no builds to execute.
Keep the environment if you want to experiment with different labels, tools, or multiple static Agents. Otherwise, stop or terminate the Agent after the lab. Stopping removes EC2 compute charges but leaves EBS storage charges; terminating the instance can remove both when the root volume is configured for deletion. Keep the Controller if you plan to continue to Part 4.
Key Takeaways
1. The Controller schedules; the Agent executes.
We have finally separated Jenkins orchestration from build compute.
2. Labels turn infrastructure into schedulable capabilities.
A Pipeline can request an execution environment instead of assuming that every node is interchangeable.
3. The Agent still runs Jenkins code through Remoting.
In our static Linux model, the Controller launches that Agent process over SSH.
4. The built-in node should no longer be build capacity.
Setting its Executor count to zero makes the Controller–Agent boundary explicit.
5. AWS permissions belong with the workload.
EC2 instance profiles and temporary credentials are preferable to long-lived AWS keys stored on the Agent or Controller.
6. Static Agents improve isolation but create static-capacity problems.
Idle cost, manual provisioning, configuration drift, fixed sizing, workspace cleanup, and host maintenance all remain.
Next: Part 4 — From Static Agents to Elastic Jenkins
The static EC2 Agent works. Builds are no longer running on the Controller, labels control workload placement, and the Agent has its own compute and permission boundary.
But a new question is now unavoidable: why should build capacity remain running when there is no build to execute?
In Part 4 — From Static Agents to Elastic Jenkins: Choosing an AWS Architecture, we will use the limitations of this static model to compare EC2-based scaling, ECS Fargate, and EKS. The goal will not be to declare one platform universally better, but to choose the right execution model for the Jenkins platform we are building in this series.
References and Version Notes
Part 3 follows the Jenkins project's guidance for distributed builds, Agent runtime Java requirements, and SSH-launched Agents.
- Jenkins - Managing Nodes
- Jenkins - Java Support Policy
- Jenkins - SSH Build Agents Plugin
The series uses Java 21 for both the Controller and Agent Jenkins processes. If you are following the series much later, verify the current Jenkins Java support policy and plugin compatibility before reproducing the setup.



















