Create CI/CD pipeline with Jenkins and Google Kubernetes Engine (GKE)

Welcome to a journey of automation and efficiency! Authors: Akash Acharya & Nishant Nath

In this comprehensive guide, we'll delve into the intricate art of constructing a robust and streamlined CI/CD pipeline. With the prowess of Jenkins, Docker Hub, and GitHub, we'll navigate the intricate pathways of software development, culminating in the seamless deployment of our applications onto the dynamic landscape of a Kubernetes Cluster hosted on Google Kubernetes Engine (GKE). Join us as we unlock the potential of automation, empowering your development endeavors with precision and agility.
‍

Quick summary of the tools

1. Jenkins: Jenkins is an open-source automation server that facilitates continuous integration (CI) and continuous delivery (CD) pipelines. It automates the building, testing, and deployment of software projects, enabling teams to rapidly deliver high-quality code changes.
2. Docker Hub: Docker Hub is a cloud-based repository service provided by Docker for storing, sharing, and managing Docker container images. It serves as a centralized platform for developers to access a vast library of pre-built Docker images and collaborate on containerized applications.
3. GitHub: GitHub is a web-based platform for version control and collaboration, primarily used for hosting Git repositories. It offers features like code review, issue tracking, and project management tools, making it a central hub for software development teams to collaborate on code, manage projects, and track changes efficiently.

Requisites:

1. We need a running Jenkins server with SSH access to it.
2. We need active GCP Project and administrator credentials for that project.

STEP - 1: Installation and configuration

1. The first step is to prepare Jenkins to work with Docker and Kubernetes. This involves installing Docker and Kubernetes command-line tools on the Jenkins cloud server and installing the Jenkins GKE plugin.
2. Install docker on the Jenkins server following this link.
3. Install the "Docker Pipeline" & "Google Kubernetes Engine" plugins in Jenkins from the 'Dashboard > Manage Jenkins > Manage Plugins page

STEP - 2: Service account and key setup

1. Jenkins will interact with the Kubernetes cluster using a Google Cloud Platform service account. For this, we need to set up the service account and give it API access.
2. Next, navigate to the "IAM & admin > Service accounts" page and create a new service account. For this demo, I have created a service account with the name "Jenkins" Assign the "Kubernetes Engine Admin" role.
3. Next, create a new JSON key for the service account. Download and save this key, as it will be needed by Jenkins.

STEP - 3: Setting up Credentials in Jenkins

1. We will use the Jenkins credentials store for the pipeline to communicate with the Kubernetes cluster and the Docker Hub registry
2. Navigate to the "Dashboard > Manage Jenkins > Manage Credentials" menu item.
3. Select the "System" sub-menu" and the "Global credentials" domain
4. Click the "Add credentials" link. Select the "Username with password" credential type and enter your Docker Hub username and password in the corresponding fields. Set the "ID" field to dockerID. Click "OK" to save the changes
5. Click the "Add credentials" link. Select the "Google Service Account from private key" credential type and set the project name (which doubles as the credential identifier) to multi-k8s. Select the "JSON key" radio button and upload the JSON key obtained in the previous step in Setting up the GCP Account. Click "OK" to

STEP- 4: Jenkins Pipeline Setup

1. Here is our GitHub repository of a sample NodeJS app
2. At this point, we are ready to start setting up our Jenkins pipeline.  
3. Log in to Jenkins and create a new Item
   1. Name: node-deploy-GKE
   2. Project Type: Pipeline
   3. Select Pipeline script and write the pipeline
   4. Replace all the variables with the values from your environment

1pipeline { 
2    agent any 
3    environment { 
4        PROJECT_ID = <Your Project ID> 
5        CLUSTER_NAME = < Your cluster name > 
6        LOCATION = < Your region name > 
7        CREDENTIALS_ID = “oceanic-granite-383005”  // service account credential name 
8    } 
9
10    stages { 
11        stage("Checkout code") { 
12            steps { 
13                git branch: 'main', url: < Your repository URL > 
14            } 
15        } 
16        stage("Build image") { 
17            steps { 
18                script { 
19                    myapp = docker.build("<docker_repo_url>/<docker_image_name>:<docker_image_tag>>") 
20                } 
21            } 
22        } 
23        
24        stage("Push image") { 
25            steps { 
26                script { 
27                    docker.withRegistry('https://registry.hub.docker.com', 'dockerhub') { 
28                            myapp.push("latest") 
29 
30                    } 
31                } 
32            } 
33        } 
34        stage('Deploy to GKE') { 
35            steps{ 
36                step([
37                    $class: 'KubernetesEngineBuilder', 
38                    projectId: env.PROJECT_ID, 
39                    clusterName: env.CLUSTER_NAME, 
40                    location: env.LOCATION, 
41                    manifestPattern: 'deployment.yml', 
42                    credentialsId: env.CREDENTIALS_ID, 
43                    verifyDeployments: true
44                ]) 
45           } 
46        } 
47    } 
48} 

4. We have our deployment.yaml file which will be used to deploy nodejs application to k8s cluster.
   1. Replace all the variables with the values from your environment‍

‍

1apiVersion: apps/v1 
2kind: Deployment 
3metadata: 
4  name: deploy 
5  labels: 
6    app: node 
7spec: 
8  replicas: 1 
9  selector: 
10    matchLabels: 
11      app: node 
12  template: 
13    metadata: 
14      labels: 
15        app: node 
16    spec: 
17      containers: 
18      - name: node 
19        image: <docker_repo_url>/<docker_image_name>:<docker_image_tag>
20        ports: 
21        - containerPort: 4000 
22        
23        
24        --- 
25 
26apiVersion: v1 
27kind: Service 
28metadata: 
29  labels: 
30    app: node 
31      name: hello 
32spec: 
33  ports: 
34  - port: 4000 
35    protocol: TCP 
36    targetPort: 4000 
37  selector: 
38    app: node 
39  sessionAffinity: None 
40  type: LoadBalancer 

‍


5. Build the pipeline in Jenkins and it should be complete with success till the Deploy to GKE step, which would indicate that it is working fine.

6. Login to the GCP console and navigate to the Kubernetes Engine > Services & Ingress. Your application should be listed there

7. Click the Endpoint link and your Web page should greet with you a Happy message.

In conclusion, by harnessing the power of Jenkins, Docker Hub, GitHub, and Google Kubernetes Engine, we've not just built a CI/CD pipeline, but we've paved the road to seamless deployment nirvana. Embrace automation, empower your development journey, and let your code take flight with the wings of innovation.
‍

Happy deploying, fellow tech enthusiasts!

‍