Day 28: Creating a DevOps Project with Jenkins Master and Agent Nodes

Jenkins Master (Server)

Jenkins’s server or master node holds all key configurations. The Jenkins master server is like a control server that orchestrates all the workflow defined in the pipelines. For example, scheduling a job, monitoring the jobs, etc.

Jenkins Agent

An agent is typically a machine or container that connects to a Jenkins master and this agent executes all the steps mentioned in a Job. When you create a Jenkins job, you have to assign an agent to it. Every agent has a label as a unique identifier.

When you trigger a Jenkins job from the master, the actual execution happens on the agent node that is configured in the job.

A single, monolithic Jenkins installation can work great for a small team with a relatively small number of projects. As your needs grow, however, it often becomes necessary to scale up. Jenkins provides a way to do this called “master to agent connection.” Instead of serving the Jenkins UI and running build jobs all on a single system, you can provide Jenkins with agents to handle the execution of jobs while the master serves the Jenkins UI and acts as a control node.

Pre-Requisites

Let’s say we’re starting with a fresh Ubuntu 22.04 Linux installation. To get an agent working make sure you install Java ( same version as Jenkins master server ) and Docker on it.

Note:- While creating an agent, be sure to separate rights, permissions, and ownership for Jenkins users.

Project

Create an agent by setting up a node on Jenkins

Create a new AWS EC2 Instance and connect it to the master(Where Jenkins is installed)

The connection of the master and agent requires SSH and the public-private key pair exchange.

Run your Job on the new agent

Use labels for the agent, your master server should trigger builds for the agent server.

Steps:

We start with creating 2 instances in AWS EC2. One for the Jenkins master and one for the Jenkins Agent.
Install Java and Jenkins on the master instance. If you don't know how to install it, you can check out the day 22 blog.
Install the Java, docker and docker-compose on your Jenkins agent. Add the USER to the docker group by using sudo usermod command.
After setting up the Jenkins master and Jenkins agent. Let's connect them via SSH.
Do cd .ssh on Jenkins master. Then do ssh-keygen on Jenkins master in the .ssh directory. It will generate the public and private key for you.

The key is generated successfully with the name id_rsa.
There will be one private key and one public key if you check with the ls command.

The id_rsa indicates the private key and id_rsa.pub indicates the public key.
Go to the agent and do cd .ssh. There will be the authorized_key folder.
Go to the Jenkins master, and do cat id_rsa.pub. It will show you the public key. Copy the Public key and paste it into the authorized_key folder of Jenkins Agent to connect them.
Now they are connected.
Go to the Jenkins server and click on set up an agent.
Write the name of your agent and check the box for permanent agent. Then click Create.
Add the description for your agent. Add the root repository. Give a label to your Agent.
Go to launch method and select launch agent via SSH.
In the host section, copy the IP from the agent instance and paste it into the host.
Then go to credentials, and click on Add. Select the kind to connect them with the private key.
Fill out the form and check on the private key (enter directly).
Then go to the Jenkins mater and do cat id_rsa for the private key. Copy the key and paste it into the private key section of the Jenkins server.

Then select the credentials you just created.
Click on Save and check if your agent is online or not by clicking on the name of the agent on the left sidebar.

The agent is successfully connected and online.
Create a new job for the same node app we used before on days 26 and 27.
write the description for your pipeline.
Select the GitHub build trigger for SCM polling from the Build trigger section.

Add the webhook for your project as we did before.
Write the pipeline for your project and mention the name of the agent you created before to run the build on an agent. This is how your pipeline will look like:

Must match the label of your agent.
Click on Save and Build Now to run the application through the agent.

Your application is up and running.
To Check on which agent is this running, go to build and click on Console Output. It will show you this:
See, it shows that the build is running on dev-node.
You successfully connect the Jenkins master with the working node. And if any change happens in the code, the build automatically runs on the agent node.

<That's all for today. Hope you like it. FOLLOW to join me in the journey of DevOps>