Runners

Jenkins Agents self-hosted setup

Configure Jenkins agent nodes for distributed builds with DevOps Hub integration


Jenkins Agents enable distributed build execution across multiple machines, allowing you to scale your CI/CD infrastructure horizontally. This guide covers setting up self-hosted Jenkins agents with DevOps Hub integration for efficient pipeline management.

Platform overview#

Jenkins follows a controller-agent architecture where the Jenkins controller (master) coordinates build execution across multiple agent nodes. This distributed approach provides several benefits:

  • Scalability - Distribute workload across multiple machines
  • Isolation - Separate build environments prevent interference
  • Specialization - Dedicated agents for specific platforms or tools
  • Performance - Parallel execution reduces overall build times
1
┌─────────────────────┐
2
│ Jenkins Controller │
3
│ (Master) │
4
│ │
5
│ ┌─────────────────┐ │
6
│ │ Job Queue │ │
7
│ │ Build History │ │
8
│ │ Agent Manager │ │
9
│ └─────────────────┘ │
10
└──────────┬──────────┘
11
│ Distributes jobs
12
13
┌──────▼──────┬─────────────┬──────────────┐
14
│ │ │ │
15
┌───▼────┐ ┌───▼────┐ ┌───▼────┐ ┌───▼────┐
16
│Agent 1 │ │Agent 2 │ │Agent 3 │ │Agent N │
17
│Linux │ │Windows │ │macOS │ │Docker │
18
│Build │ │Build │ │Build │ │Cloud │
19
└────────┘ └────────┘ └────────┘ └────────┘

Prerequisites#

Before setting up Jenkins agents, ensure you have:

Jenkins Controller Setup#

  • Jenkins LTS version 2.401+ running and accessible
  • Administrative access to Jenkins controller
  • Network connectivity between controller and agent machines
  • Java 11 or 17 installed on controller

Agent Infrastructure Requirements#

  • Compute resources appropriate for build workloads
  • Operating system compatible with your build requirements
  • Network access to Jenkins controller (typically port 8080)
  • Java Runtime Environment 11 or 17
  • Build tools required for your projects (Maven, Gradle, Node.js, etc.)

DevOps Hub Integration#

  • DevOps Hub API token with project access permissions
  • Project ID for DevOps Hub integration
  • Branch management permissions for environment creation

Infrastructure setup#

1. Prepare agent machines#

For optimal performance, provision dedicated machines for Jenkins agents:

Minimum specifications:

  • CPU: 2+ cores per concurrent build
  • RAM: 4GB+ per concurrent build
  • Storage: 50GB+ with fast I/O
  • Network: Reliable connection to Jenkins controller

Recommended setup for production:

1
# Ubuntu/Debian agent setup
2
sudo apt update && sudo apt upgrade -y
3
4
# Install Java 17 (recommended)
5
sudo apt install openjdk-17-jre-headless -y
6
7
# Verify Java installation
8
java -version
9
10
# Install build tools
11
sudo apt install -y git curl wget unzip
12
sudo apt install -y build-essential # For C/C++ compilation

2. Network configuration#

Ensure proper network connectivity between controller and agents:

Required ports:

  • SSH agents: Port 22 (SSH)
  • JNLP agents: Random port assigned by controller
  • Inbound agents: Port defined in agent configuration

Firewall configuration:

1
# Allow SSH access (for SSH agents)
2
sudo ufw allow ssh
3
4
# Allow Jenkins controller access
5
sudo ufw allow from JENKINS_CONTROLLER_IP
6
7
# Enable firewall
8
sudo ufw --force enable

Agent installation#

SSH agents (Linux/macOS)#

SSH agents use SSH protocol for secure communication with the Jenkins controller.

1. Create Jenkins user#

1
# Create dedicated user for Jenkins
2
sudo adduser jenkins
3
4
# Add to docker group (if Docker builds needed)
5
sudo usermod -a -G docker jenkins
6
7
# Switch to jenkins user
8
sudo su - jenkins

2. SSH key setup#

1
# Generate SSH key pair on Jenkins controller
2
ssh-keygen -t rsa -b 4096 -f ~/.ssh/jenkins_agent_key
3
4
# Copy public key to agent
5
ssh-copy-id -i ~/.ssh/jenkins_agent_key.pub jenkins@AGENT_IP

3. Configure SSH agent in Jenkins#

  1. Navigate to Manage Jenkins > Manage Nodes and Clouds
  2. Click New Node
  3. Enter node name and select Permanent Agent
  4. Configure agent settings:
1
Name: linux-agent-01
2
Description: Linux build agent for DevOps Hub projects
3
Number of executors: 2
4
Remote root directory: /home/jenkins/jenkins-agent
5
Labels: linux ubuntu docker devops-hub
6
Usage: Use this node as much as possible
7
Launch method: Launch agents via SSH
8
Host: AGENT_IP
9
Credentials: [Select SSH private key credential]
10
Host Key Verification Strategy: Known hosts file Verification Strategy
11
Availability: Keep this agent online as much as possible

JNLP agents (Java Web Start)#

JNLP agents connect outbound from agent to controller, useful when SSH isn't available.

1. Create JNLP agent#

  1. In Jenkins, create a new Permanent Agent
  2. Set Launch method to Launch agent by connecting it to the controller
  3. Save the configuration to generate the agent secret

2. Download and start agent#

1
# Download agent.jar from Jenkins controller
2
wget http://JENKINS_CONTROLLER:8080/jnlpJars/agent.jar
3
4
# Start agent with secret
5
java -jar agent.jar -jnlpUrl http://JENKINS_CONTROLLER:8080/computer/AGENT_NAME/slave-agent.jnlp -secret AGENT_SECRET -workDir "/home/jenkins/jenkins-agent"

3. Run as system service#

Create systemd service for automatic startup:

1
# Create service file
2
sudo tee /etc/systemd/system/jenkins-agent.service > /dev/null <<EOF
3
[Unit]
4
Description=Jenkins Agent
5
After=network.target
6
7
[Service]
8
Type=simple
9
User=jenkins
10
WorkingDirectory=/home/jenkins
11
ExecStart=/usr/bin/java -jar /home/jenkins/agent.jar -jnlpUrl http://JENKINS_CONTROLLER:8080/computer/AGENT_NAME/slave-agent.jnlp -secret AGENT_SECRET -workDir "/home/jenkins/jenkins-agent"
12
Restart=always
13
RestartSec=10
14
15
[Install]
16
WantedBy=multi-user.target
17
EOF
18
19
# Enable and start service
20
sudo systemctl enable jenkins-agent
21
sudo systemctl start jenkins-agent
22
sudo systemctl status jenkins-agent

Configuration#

Node management and labels#

Labels help route specific jobs to appropriate agents:

Common label patterns:

  • Platform labels: linux, windows, macos
  • Tool labels: docker, nodejs, python, maven
  • Environment labels: dev, staging, prod
  • Capability labels: gpu, high-memory, ssd

Example agent configuration:

1
Labels: linux ubuntu-20.04 docker nodejs python maven devops-hub

Executors and concurrency#

Configure appropriate number of executors based on agent resources:

1
Number of executors = CPU cores / 2 (for I/O bound builds)
2
Number of executors = CPU cores (for CPU bound builds)

Workspace management#

Set up clean workspace policies:

1
pipeline {
2
agent { label 'devops-hub' }
3
options {
4
// Clean workspace before build
5
skipDefaultCheckout()
6
// Keep only last 10 builds
7
buildDiscarder(logRotator(numToKeepStr: '10'))
8
}
9
stages {
10
stage('Checkout') {
11
steps {
12
cleanWs()
13
checkout scm
14
}
15
}
16
}
17
}

DevOps Hub integration#

Credential management#

Store DevOps Hub credentials securely in Jenkins:

  1. Navigate to Manage Jenkins > Manage Credentials
  2. Add Secret text credential for DevOps Hub API token
  3. Add Username with password for DevOps Hub project access
1
// Access credentials in pipeline
2
environment {
3
DEVOPS_HUB_TOKEN = credentials('devops-hub-api-token')
4
PROJECT_ID = credentials('devops-hub-project-id')
5
DEVOPS_HUB_URL = 'https://console.assistance.bg/api/v2'
6
}

Pipeline integration#

Complete Jenkins pipeline with DevOps Hub integration:

1
pipeline {
2
agent { label 'devops-hub' }
3
4
environment {
5
DEVOPS_HUB_TOKEN = credentials('devops-hub-api-token')
6
PROJECT_ID = credentials('devops-hub-project-id')
7
DEVOPS_HUB_URL = 'https://console.assistance.bg/api/v2'
8
BRANCH_NAME = "jenkins-build-${BUILD_NUMBER}"
9
DATABASE_URL = ""
10
}
11
12
options {
13
buildDiscarder(logRotator(numToKeepStr: '10'))
14
timeout(time: 30, unit: 'MINUTES')
15
}
16
17
stages {
18
stage('Setup Environment') {
19
steps {
20
script {
21
// Create DevOps Hub branch for this build
22
def response = sh(
23
script: """
24
curl -s -X POST "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/branches" \
25
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
26
-H "Content-Type: application/json" \
27
-d '{"branch": {"name": "${BRANCH_NAME}", "parent_id": "main"}}'
28
""",
29
returnStdout: true
30
).trim()
31
32
def json = readJSON text: response
33
env.DATABASE_URL = json.branch.database_url
34
35
echo "Created DevOps Hub branch: ${BRANCH_NAME}"
36
echo "Database URL: ${DATABASE_URL}"
37
}
38
}
39
}
40
41
stage('Checkout') {
42
steps {
43
checkout scm
44
}
45
}
46
47
stage('Install Dependencies') {
48
when {
49
anyOf {
50
changeset "package*.json"
51
not { fileExists 'node_modules' }
52
}
53
}
54
steps {
55
sh '''
56
npm ci --prefer-offline --no-audit
57
'''
58
}
59
}
60
61
stage('Database Setup') {
62
steps {
63
script {
64
// Run database migrations
65
sh """
66
export DATABASE_URL="${DATABASE_URL}"
67
npm run migrate:deploy
68
"""
69
}
70
}
71
}
72
73
stage('Run Tests') {
74
parallel {
75
stage('Unit Tests') {
76
steps {
77
sh '''
78
export DATABASE_URL="${DATABASE_URL}"
79
npm run test:unit -- --coverage --ci
80
'''
81
}
82
post {
83
always {
84
publishTestResults testResultsPattern: 'test-results.xml'
85
publishCoverageResults([
86
[
87
path: 'coverage/clover.xml',
88
type: 'clover'
89
]
90
])
91
}
92
}
93
}
94
95
stage('Integration Tests') {
96
steps {
97
sh '''
98
export DATABASE_URL="${DATABASE_URL}"
99
npm run test:integration -- --ci
100
'''
101
}
102
}
103
104
stage('Linting') {
105
steps {
106
sh '''
107
npm run lint -- --format=checkstyle --output-file=lint-results.xml
108
'''
109
}
110
post {
111
always {
112
recordIssues(
113
enabledForFailure: true,
114
aggregatingResults: false,
115
tools: [checkStyle(pattern: 'lint-results.xml')]
116
)
117
}
118
}
119
}
120
}
121
}
122
123
stage('Build Application') {
124
steps {
125
sh '''
126
npm run build
127
'''
128
129
// Archive build artifacts
130
archiveArtifacts artifacts: 'dist/**', fingerprint: true
131
}
132
}
133
134
stage('Deploy to Staging') {
135
when {
136
anyOf {
137
branch 'main'
138
branch 'develop'
139
}
140
}
141
steps {
142
script {
143
// Deploy using DevOps Hub branch
144
sh """
145
export DATABASE_URL="${DATABASE_URL}"
146
npm run deploy:staging
147
"""
148
149
// Store deployment info
150
def deploymentUrl = sh(
151
script: "echo 'https://${BRANCH_NAME}.staging.assistance.bg'",
152
returnStdout: true
153
).trim()
154
155
// Add deployment status to DevOps Hub
156
sh """
157
curl -X PATCH "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/branches/${BRANCH_NAME}" \
158
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
159
-H "Content-Type: application/json" \
160
-d '{"branch": {"metadata": {"deployment_url": "${deploymentUrl}", "jenkins_build": "${BUILD_NUMBER}"}}}'
161
"""
162
163
echo "Deployed to: ${deploymentUrl}"
164
}
165
}
166
}
167
}
168
169
post {
170
always {
171
// Clean up DevOps Hub branch for short-lived builds
172
script {
173
if (env.BRANCH_NAME && env.BRANCH_NAME.startsWith('jenkins-build-')) {
174
sh """
175
curl -X DELETE "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/branches/${BRANCH_NAME}" \
176
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}"
177
"""
178
echo "Cleaned up DevOps Hub branch: ${BRANCH_NAME}"
179
}
180
}
181
}
182
183
success {
184
// Notify success
185
script {
186
if (env.CHANGE_ID) {
187
// This is a PR build
188
sh """
189
curl -X POST "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/notifications" \
190
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
191
-H "Content-Type: application/json" \
192
-d '{
193
"type": "build_success",
194
"message": "✅ Jenkins build #${BUILD_NUMBER} succeeded for PR #${CHANGE_ID}",
195
"metadata": {
196
"build_number": "${BUILD_NUMBER}",
197
"pr_number": "${CHANGE_ID}",
198
"branch": "${CHANGE_BRANCH}"
199
}
200
}'
201
"""
202
}
203
}
204
}
205
206
failure {
207
// Notify failure
208
sh """
209
curl -X POST "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/notifications" \
210
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
211
-H "Content-Type: application/json" \
212
-d '{
213
"type": "build_failure",
214
"message": "❌ Jenkins build #${BUILD_NUMBER} failed on ${BRANCH_NAME}",
215
"metadata": {
216
"build_number": "${BUILD_NUMBER}",
217
"branch": "${BRANCH_NAME}",
218
"failure_stage": "${env.FAILED_STAGE}"
219
}
220
}'
221
"""
222
}
223
}
224
}

Cloud provisioning#

Docker agents#

Use Docker for ephemeral, clean build environments:

1. Install Docker plugin#

  1. Navigate to Manage Jenkins > Manage Plugins
  2. Install Docker Plugin
  3. Restart Jenkins

2. Configure Docker cloud#

1
// Docker agent template
2
pipeline {
3
agent {
4
docker {
5
image 'node:18-alpine'
6
args '-v /var/run/docker.sock:/var/run/docker.sock'
7
}
8
}
9
10
stages {
11
stage('Build') {
12
steps {
13
sh 'npm install && npm run build'
14
}
15
}
16
}
17
}

3. Dynamic Docker agents#

Configure dynamic Docker agent provisioning:

  1. Manage Jenkins > Manage Nodes and Clouds > Configure Clouds
  2. Add Docker Cloud
  3. Configure Docker host URL: unix:///var/run/docker.sock
  4. Add Docker templates for different environments

Kubernetes agents#

For scalable cloud-native builds:

1. Install Kubernetes plugin#

1
# Install Kubernetes plugin
2
# Manage Jenkins > Manage Plugins > Available > Kubernetes

2. Configure Kubernetes cloud#

1
# kubernetes-agent-template.yaml
2
apiVersion: v1
3
kind: Pod
4
metadata:
5
labels:
6
jenkins: agent
7
spec:
8
containers:
9
- name: jnlp
10
image: jenkins/inbound-agent:latest
11
env:
12
- name: JENKINS_URL
13
value: "http://jenkins-controller:8080"
14
- name: build
15
image: node:18-alpine
16
command:
17
- cat
18
tty: true
19
volumeMounts:
20
- name: docker-sock
21
mountPath: /var/run/docker.sock
22
volumes:
23
- name: docker-sock
24
hostPath:
25
path: /var/run/docker.sock

3. Use Kubernetes agents in pipeline#

1
pipeline {
2
agent {
3
kubernetes {
4
yaml """
5
apiVersion: v1
6
kind: Pod
7
spec:
8
containers:
9
- name: build
10
image: node:18-alpine
11
command:
12
- cat
13
tty: true
14
- name: docker
15
image: docker:dind
16
securityContext:
17
privileged: true
18
"""
19
}
20
}
21
22
stages {
23
stage('Build') {
24
steps {
25
container('build') {
26
sh 'npm install && npm run build'
27
}
28
}
29
}
30
31
stage('Docker Build') {
32
steps {
33
container('docker') {
34
sh 'docker build -t myapp:${BUILD_NUMBER} .'
35
}
36
}
37
}
38
}
39
}

AWS EC2 agents#

Dynamic EC2 instance provisioning:

1. Install EC2 plugin#

  1. Install Amazon EC2 Plugin
  2. Configure AWS credentials
  3. Set up EC2 cloud configuration

2. EC2 agent template#

1
// EC2 agent configuration
2
AMI ID: ami-0c55b159cbfafe1d0 // Ubuntu 20.04 LTS
3
Instance Type: t3.medium
4
Security Group: jenkins-agents (ports 22, 8080)
5
Key Pair: jenkins-ec2-key
6
User Data: |
7
#!/bin/bash
8
apt-get update
9
apt-get install -y openjdk-17-jre-headless
10
# Additional setup scripts

Multi-OS support#

Windows agents#

1. Install Java on Windows#

1
# Install Java using Chocolatey
2
choco install openjdk17 -y
3
4
# Verify installation
5
java -version

2. Configure Windows agent#

1
# Create jenkins user
2
net user jenkins PASSWORD /add
3
net localgroup administrators jenkins /add
4
5
# Create agent directory
6
mkdir C:\jenkins
7
8
# Download and run agent
9
Invoke-WebRequest -Uri "http://JENKINS_CONTROLLER:8080/jnlpJars/agent.jar" -OutFile "C:\jenkins\agent.jar"
10
11
# Run as Windows service using NSSM
12
nssm install JenkinsAgent
13
nssm set JenkinsAgent Application "java"
14
nssm set JenkinsAgent AppParameters "-jar C:\jenkins\agent.jar -jnlpUrl http://JENKINS_CONTROLLER:8080/computer/windows-agent/slave-agent.jnlp -secret SECRET -workDir C:\jenkins"
15
nssm start JenkinsAgent

macOS agents#

1. Install dependencies#

1
# Install Homebrew (if not installed)
2
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
3
4
# Install Java
5
brew install openjdk@17
6
7
# Create symlink
8
sudo ln -sfn /opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-17.jdk

2. Configure macOS agent#

1
# Create jenkins user
2
sudo dscl . -create /Users/jenkins
3
sudo dscl . -create /Users/jenkins UserShell /bin/bash
4
sudo dscl . -create /Users/jenkins RealName "Jenkins Agent"
5
sudo dscl . -create /Users/jenkins UniqueID 1001
6
sudo dscl . -create /Users/jenkins PrimaryGroupID 20
7
sudo dscl . -create /Users/jenkins NFSHomeDirectory /Users/jenkins
8
9
# Create home directory
10
sudo createhomedir -c -u jenkins
11
12
# Set up SSH keys or JNLP agent as described earlier

Cross-platform pipeline#

1
pipeline {
2
stages {
3
stage('Build Matrix') {
4
matrix {
5
axes {
6
axis {
7
name 'PLATFORM'
8
values 'linux', 'windows', 'macos'
9
}
10
}
11
stages {
12
stage('Platform Build') {
13
agent { label "${PLATFORM}" }
14
steps {
15
script {
16
if (PLATFORM == 'windows') {
17
bat 'npm install && npm run build'
18
} else {
19
sh 'npm install && npm run build'
20
}
21
}
22
}
23
}
24
}
25
}
26
}
27
}
28
}

Test pipeline#

Complete test pipeline#

Create a comprehensive test pipeline to validate your Jenkins agent setup:

1
pipeline {
2
agent none
3
4
environment {
5
DEVOPS_HUB_TOKEN = credentials('devops-hub-api-token')
6
PROJECT_ID = credentials('devops-hub-project-id')
7
DEVOPS_HUB_URL = 'https://console.assistance.bg/api/v2'
8
}
9
10
stages {
11
stage('Agent Connectivity Test') {
12
parallel {
13
stage('Linux Agent Test') {
14
agent { label 'linux' }
15
steps {
16
sh '''
17
echo "Testing Linux agent..."
18
uname -a
19
java -version
20
docker --version
21
git --version
22
node --version
23
npm --version
24
'''
25
}
26
}
27
28
stage('Windows Agent Test') {
29
agent { label 'windows' }
30
when {
31
expression {
32
return Jenkins.instance.nodes.find { it.labelString.contains('windows') } != null
33
}
34
}
35
steps {
36
bat '''
37
echo "Testing Windows agent..."
38
systeminfo | findstr /B "OS Name OS Version"
39
java -version
40
git --version
41
node --version
42
npm --version
43
'''
44
}
45
}
46
}
47
}
48
49
stage('DevOps Hub Integration Test') {
50
agent { label 'devops-hub' }
51
steps {
52
script {
53
def testBranch = "jenkins-test-${BUILD_NUMBER}"
54
55
// Test branch creation
56
def createResponse = sh(
57
script: """
58
curl -s -w "\\n%{http_code}" -X POST "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/branches" \
59
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
60
-H "Content-Type: application/json" \
61
-d '{"branch": {"name": "${testBranch}"}}'
62
""",
63
returnStdout: true
64
).trim().split('\n')
65
66
def statusCode = createResponse[-1]
67
def responseBody = createResponse[0..-2].join('\n')
68
69
if (statusCode == '201' || statusCode == '200') {
70
echo "✅ DevOps Hub branch creation: SUCCESS"
71
72
def json = readJSON text: responseBody
73
def databaseUrl = json.branch.database_url
74
echo "Database URL: ${databaseUrl}"
75
76
// Test database connection
77
sh """
78
export DATABASE_URL="${databaseUrl}"
79
timeout 10 bash -c 'until pg_isready -d "\${DATABASE_URL}"; do sleep 1; done'
80
echo "✅ Database connection: SUCCESS"
81
"""
82
83
// Cleanup test branch
84
sh """
85
curl -s -X DELETE "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/branches/${testBranch}" \
86
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}"
87
"""
88
echo "✅ DevOps Hub branch cleanup: SUCCESS"
89
} else {
90
error("❌ DevOps Hub integration failed with status ${statusCode}")
91
}
92
}
93
}
94
}
95
96
stage('Build Tools Test') {
97
agent { label 'devops-hub' }
98
steps {
99
sh '''
100
echo "Testing build tools..."
101
102
# Test Docker
103
docker run --rm hello-world
104
echo "✅ Docker: SUCCESS"
105
106
# Test npm/node
107
npm --version
108
node --version
109
echo "✅ Node.js/npm: SUCCESS"
110
111
# Test git
112
git --version
113
echo "✅ Git: SUCCESS"
114
115
# Test curl
116
curl --version
117
echo "✅ Curl: SUCCESS"
118
'''
119
}
120
}
121
122
stage('Performance Test') {
123
agent { label 'devops-hub' }
124
steps {
125
script {
126
def startTime = System.currentTimeMillis()
127
128
sh '''
129
# CPU test
130
echo "Testing CPU performance..."
131
time dd if=/dev/zero of=/tmp/test bs=1M count=100
132
133
# Memory test
134
echo "Testing memory..."
135
free -h
136
137
# Disk I/O test
138
echo "Testing disk I/O..."
139
time sync
140
141
# Network test
142
echo "Testing network..."
143
ping -c 3 google.com
144
145
# Clean up
146
rm -f /tmp/test
147
'''
148
149
def endTime = System.currentTimeMillis()
150
def duration = (endTime - startTime) / 1000
151
echo "✅ Performance test completed in ${duration} seconds"
152
}
153
}
154
}
155
}
156
157
post {
158
always {
159
script {
160
// Send test results to DevOps Hub
161
def testStatus = currentBuild.result == null ? 'SUCCESS' : currentBuild.result
162
def testReport = [
163
type: 'jenkins_agent_test',
164
status: testStatus,
165
build_number: BUILD_NUMBER,
166
duration: currentBuild.duration,
167
agents_tested: Jenkins.instance.nodes.findAll {
168
it.labelString.contains('devops-hub')
169
}.size()
170
]
171
172
sh """
173
curl -X POST "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/test-reports" \
174
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
175
-H "Content-Type: application/json" \
176
-d '${groovy.json.JsonOutput.toJson(testReport)}'
177
"""
178
}
179
}
180
}
181
}

Production deployment#

Agent pool management#

Implement agent pools for different workload types:

1. Create agent pools#

1
// Agent pool configurations
2
def agentPools = [
3
'build-pool': [
4
'labels': 'linux docker npm maven',
5
'capacity': 4,
6
'instance_type': 't3.large'
7
],
8
'test-pool': [
9
'labels': 'linux docker selenium cypress',
10
'capacity': 2,
11
'instance_type': 't3.xlarge'
12
],
13
'deploy-pool': [
14
'labels': 'linux aws kubectl terraform',
15
'capacity': 1,
16
'instance_type': 't3.medium'
17
]
18
]

2. Pool-aware pipeline#

1
pipeline {
2
stages {
3
stage('Build') {
4
agent { label 'build-pool && linux' }
5
steps {
6
sh 'npm run build'
7
}
8
}
9
10
stage('Test') {
11
agent { label 'test-pool && docker' }
12
steps {
13
sh 'npm run test:e2e'
14
}
15
}
16
17
stage('Deploy') {
18
agent { label 'deploy-pool && aws' }
19
steps {
20
sh 'terraform apply -auto-approve'
21
}
22
}
23
}
24
}

Monitoring and metrics#

1. Jenkins monitoring#

1
// Monitor agent health
2
pipeline {
3
agent { label 'monitoring' }
4
triggers {
5
cron('*/5 * * * *') // Every 5 minutes
6
}
7
8
stages {
9
stage('Agent Health Check') {
10
steps {
11
script {
12
def agents = Jenkins.instance.nodes
13
def healthReport = [:]
14
15
agents.each { agent ->
16
def status = agent.toComputer().isOnline() ? 'online' : 'offline'
17
def lastSeen = agent.toComputer().getLastConnectTime()
18
19
healthReport[agent.name] = [
20
status: status,
21
lastSeen: lastSeen,
22
labels: agent.labelString,
23
executors: agent.numExecutors
24
]
25
}
26
27
// Send metrics to DevOps Hub
28
sh """
29
curl -X POST "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/metrics" \
30
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
31
-H "Content-Type: application/json" \
32
-d '{"agent_health": ${groovy.json.JsonOutput.toJson(healthReport)}}'
33
"""
34
}
35
}
36
}
37
}
38
}

2. Resource utilization monitoring#

1
# Agent monitoring script (deploy to each agent)
2
#!/bin/bash
3
# /usr/local/bin/monitor-agent.sh
4
5
AGENT_NAME=$(hostname)
6
JENKINS_CONTROLLER="http://jenkins:8080"
7
8
while true; do
9
# Collect metrics
10
CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d'%' -f1)
11
MEMORY_USAGE=$(free | grep Mem | awk '{printf "%.2f", $3/$2 * 100.0}')
12
DISK_USAGE=$(df -h / | awk 'NR==2{print $5}' | cut -d'%' -f1)
13
14
# Send to Jenkins or monitoring system
15
curl -X POST "$JENKINS_CONTROLLER/agent-metrics" \
16
-H "Content-Type: application/json" \
17
-d "{
18
\"agent\": \"$AGENT_NAME\",
19
\"cpu_usage\": $CPU_USAGE,
20
\"memory_usage\": $MEMORY_USAGE,
21
\"disk_usage\": $DISK_USAGE,
22
\"timestamp\": $(date +%s)
23
}"
24
25
sleep 60
26
done

Blue Ocean UI integration#

Enhance pipeline visualization with Blue Ocean:

1. Install Blue Ocean#

  1. Manage Jenkins > Manage Plugins
  2. Install Blue Ocean plugin suite
  3. Restart Jenkins

2. Configure Blue Ocean pipeline#

1
// Jenkinsfile optimized for Blue Ocean
2
pipeline {
3
agent none
4
5
stages {
6
stage('Prepare') {
7
agent { label 'devops-hub' }
8
steps {
9
echo "🚀 Starting DevOps Hub build pipeline"
10
script {
11
currentBuild.displayName = "#${BUILD_NUMBER} - ${env.BRANCH_NAME}"
12
}
13
}
14
}
15
16
stage('Parallel Build & Test') {
17
parallel {
18
stage('🔨 Build Application') {
19
agent { label 'build-pool' }
20
steps {
21
checkout scm
22
sh 'npm ci'
23
sh 'npm run build'
24
archiveArtifacts artifacts: 'dist/**'
25
}
26
}
27
28
stage('🧪 Unit Tests') {
29
agent { label 'test-pool' }
30
steps {
31
checkout scm
32
sh 'npm ci'
33
sh 'npm run test:unit'
34
publishTestResults testResultsPattern: 'test-results.xml'
35
}
36
}
37
38
stage('🐳 Docker Build') {
39
agent { label 'docker' }
40
steps {
41
checkout scm
42
sh 'docker build -t app:${BUILD_NUMBER} .'
43
}
44
}
45
}
46
}
47
48
stage('🚀 Deploy to Staging') {
49
agent { label 'deploy-pool' }
50
when {
51
anyOf {
52
branch 'main'
53
branch 'develop'
54
}
55
}
56
steps {
57
echo "Deploying to staging environment"
58
sh 'kubectl apply -f k8s/staging/'
59
}
60
}
61
}
62
63
post {
64
always {
65
script {
66
if (env.BRANCH_NAME == 'main') {
67
// Send deployment notification
68
sh """
69
curl -X POST "${DEVOPS_HUB_URL}/projects/${PROJECT_ID}/deployments" \
70
-H "Authorization: Bearer ${DEVOPS_HUB_TOKEN}" \
71
-H "Content-Type: application/json" \
72
-d '{
73
"environment": "staging",
74
"status": "${currentBuild.result ?: 'SUCCESS'}",
75
"build_number": "${BUILD_NUMBER}",
76
"commit_sha": "${env.GIT_COMMIT}"
77
}'
78
"""
79
}
80
}
81
}
82
}
83
}

High availability setup#

For production environments, implement agent redundancy:

1. Agent failover configuration#

1
// Multi-agent deployment strategy
2
def deployToAgents(agents, task) {
3
def deployments = [:]
4
5
agents.each { agent ->
6
deployments["Deploy to ${agent}"] = {
7
node(agent) {
8
try {
9
task()
10
} catch (Exception e) {
11
echo "Deployment failed on ${agent}: ${e.message}"
12
// Continue with other agents
13
}
14
}
15
}
16
}
17
18
parallel deployments
19
}
20
21
// Usage in pipeline
22
stage('Resilient Deployment') {
23
steps {
24
script {
25
def productionAgents = ['prod-agent-1', 'prod-agent-2', 'prod-agent-3']
26
27
deployToAgents(productionAgents) {
28
sh 'kubectl apply -f k8s/production/'
29
sh 'kubectl rollout status deployment/app'
30
}
31
}
32
}
33
}

2. Load balancer configuration#

1
# nginx.conf for Jenkins controller load balancing
2
upstream jenkins_controllers {
3
server jenkins-master-1:8080 max_fails=3 fail_timeout=30s;
4
server jenkins-master-2:8080 max_fails=3 fail_timeout=30s backup;
5
}
6
7
server {
8
listen 80;
9
server_name jenkins.assistance.bg;
10
11
location / {
12
proxy_pass http://jenkins_controllers;
13
proxy_set_header Host $host;
14
proxy_set_header X-Real-IP $remote_addr;
15
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
16
proxy_set_header X-Forwarded-Proto $scheme;
17
}
18
}

This comprehensive Jenkins Agents setup provides a robust foundation for distributed CI/CD with DevOps Hub integration. The configuration supports multiple agent types, dynamic provisioning, cross-platform builds, and production-grade monitoring and high availability.