Building CI/CD Pipeline with Jenkins, Github and Docker Swarm

Overview

In this blog we will cover how to build an fully automated software deployment pipeline starting from developer committing code to Github till it gets deployed in Production. We will use Docker swarm cluster for deployment. I have used sample Go based application (Thanks to krasi-georgiev). Github as code repository and Jenkins as our build and automation tool.

Infrastructure Components

• Docker Swarm Cluster
• Jenkins
• Github a/c
• Docker Hub a/c – Pushing the images to repo (Can be private repo)

Quick way to setup 3 node (1xMaster and 2xWorker) Docker Swarm cluster

Run below command on all 3 modes

  sudo wget https://get.docker.com/builds/Linux/x86_64/docker-17.05.0-ce.tgz

  sudo tar -xvf docker-17.05.0-ce.tgz

  sudo cp docker/docker* /usr/bin/


cat << EOF > docker.service 

   [Unit]

   Description=Docker Application Container Engine

   Documentation=http://docs.docker.io

   [Service]

   ExecStart=/usr/bin/docker daemon

   --iptables=false

   --ip-masq=false

   --host=unix:///var/run/docker.sock

   --log-level=error

   --storage-driver=overlay2

   Restart=on-failure

   RestartSec=5

   [Install]

   WantedBy=multi-user.target

EOF


   sudo mv docker.service /etc/systemd/system/

   sudo systemctl daemon-reload

   sudo systemctl enable docker

   sudo systemctl start docker

   sleep 2

   sudo docker version

Run below command on Master node

ubuntu@docker-master-1:~$ sudo docker swarm init --advertise-addr 172.16.10.80

ubuntu@docker-master-1:~$ sudo docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
62jshiksy9o1b8z2fs1do8t11 * docker-master-1 Ready Active Leader

ubuntu@docker-master-1:~$ sudo docker swarm join-token worker
To add a worker to this swarm, run the following command:

docker swarm join \
 --token SWMTKN-1-1e85luyht07enmdu2qdu4qsm7ud679wjk77d39wiuzreqfopqd-ecrjma1qostzhae7ws1u5e8h9 \
 172.16.10.80:2377

Run below command on all the worker nodes to join the swarm cluster

ubuntu@docker-swarm-4:~$ sudo docker swarm join \
> --token SWMTKN-1-1e85luyht07enmdu2qdu4qsm7ud679wjk77d39wiuzreqfopqd-ecrjma1qostzhae7ws1u5e8h9 \
> 172.16.10.80:2377
sudo: unable to resolve host docker-swarm-4: Connection timed out
This node joined a swarm as a worker.
ubuntu@docker-swarm-4:~$

ubuntu@docker-master-1:~$ sudo docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
62jshiksy9o1b8z2fs1do8t11 * docker-master-1 Ready Active Leader
s79uzkl4izd57j22b67gvibe9 docker-swarm-4 Ready Active
v9zji36exkwce4nhrb6s1kktp docker-swarm-3 Ready Active

ubuntu@docker-master-1:~$


ubuntu@docker-master-1:~$ sudo docker info

Containers: 10

Running: 5

Paused: 0

Stopped: 5

Images: 9

Server Version: 17.05.0-ce

Storage Driver: aufs

Root Dir: /var/lib/docker/aufs

Backing Filesystem: extfs

Dirs: 110

Dirperm1 Supported: true

Logging Driver: json-file

Cgroup Driver: cgroupfs

Plugins:

Volume: local

Network: bridge host macvlan null overlay

Swarm: active

NodeID: 62jshiksy9o1b8z2fs1do8t11

Is Manager: true

ClusterID: cxa1henbpq5lbsjlcvaeqzzaa

Managers: 1

Nodes: 2

Orchestration:

  Task History Retention Limit: 5

Raft:

  Snapshot Interval: 10000

  Number of Old Snapshots to Retain: 0

  Heartbeat Tick: 1

  Election Tick: 3

Dispatcher:

  Heartbeat Period: 5 seconds

CA Configuration:

  Expiry Duration: 3 months

Node Address: 172.16.10.80

Manager Addresses:

  172.16.10.80:2377

Runtimes: runc

Default Runtime: runc

Init Binary: docker-init

containerd version: 9048e5e50717ea4497b757314bad98ea3763c145

runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228

init version: 949e6fa

Security Options:

apparmor

seccomp

  Profile: default

Kernel Version: 4.4.0-101-generic

Operating System: Ubuntu 16.04.3 LTS

OSType: linux

Architecture: x86_64

CPUs: 4

Total Memory: 3.859GiB

Name: docker-master-1

ID: 6D6M:P6I2:2SXI:UEN3:6IIT:G4LR:SGS2:BAEO:PHMA:OVHF:OJCL:PABI

Docker Root Dir: /var/lib/docker

Debug Mode (client): false

Debug Mode (server): false

Registry: https://index.docker.io/v1/

Experimental: false

Insecure Registries:

127.0.0.0/8

Live Restore Enabled: false

Create a Jenkins Container and add Github and Dockerhub credentials to it

sudo docker stack deploy -c jenkins.yml jenkins 

ubuntu@docker-master-1:~$ cat jenkins.yml
version: '3'
services:
main:
 image: jenkinsci/jenkins:${TAG:-lts-alpine}
 ports:
 - ${UI_PORT:-8080}:8080
 - ${AGENTS_PORT:-50000}:50000
 environment:
 - JENKINS_OPTS="--prefix=/jenkins"
ubuntu@docker-master-1:~$

Login to Jenkins (http://swarm-master-ip:8080/jenkins). First time when you login to Jenkins it will ask for a secret, please run below command to get the secret

docker service logs jenkins_main

Jenkins Plugin needed

• Pipeline
• GitHub plugin
• Self-Organizing Swarm Plug-in Modules

Install the Jenkins swarm agent on all the docker nodes to make them Jenkins slave. With this agent installed these swarm nodes will be auto discovered on Jenkins master

Run below commands on swarm master to create a global service which will install Jenkins swarm agent on all the nodes. when this service is created Docker nodes will be discovered on Jenkins master as slave nodes

Create a Docker secret which will be used be Jenkins swarm agent to connect to jenkins master

sudo echo "-master http://10.9.81.107:8080/jenkins -password admin -username admin"| sudo docker secret create jenkins-v1 -

docker service create \

    --mode=global \

    --name jenkins-swarm-agent \

    -e LABELS=docker-prod \

    --mount "type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock" \

    --mount "type=bind,source=/tmp/,target=/tmp/" \

    --secret source=jenkins-v1,target=jenkins \

    vfabric/jenkins-swarm-agent

Create a new GitHub organization project on Jenkins master which will scan the Github a/c and download code and start executing the pipeline . This plugin will look for repo which has Jenkinsfile and clone that repo. we can also specify the repo name in the project configuration.

Started
[Thu Dec 07 01:23:55 GMT 2017] Starting organization scan...
[Thu Dec 07 01:23:55 GMT 2017] Updating actions...
Looking up details of sbikram...
Organization URL: https://github.com/sbikram
[Thu Dec 07 01:23:56 GMT 2017] Consulting GitHub Organization
01:23:56 Connecting to https://api.github.com using sbikram/******
01:23:57 Looking up repositories of myself sbikram
Proposing ansible-playbook-vmwaretools
Examining sbikram/ansible-playbook-vmwaretools

  Checking branches...

  Getting remote branches...

    Checking branch master

  Getting remote pull requests...
      ‘Jenkinsfile’ not found
    Does not meet criteria

  1 branches were processed

  Checking pull-requests...

  0 pull requests were processed

Finished examining sbikram/ansible-playbook-vmwaretools

Proposing docker-ci-cd
01:24:00 Connecting to https://api.github.com using sbikram/******
Examining sbikram/docker-ci-cd

  Checking branches...

  Getting remote branches...

    Checking branch master

  Getting remote pull requests...
      ‘Jenkinsfile’ found
    Met criteria

  1 branches were processed (query completed)

  1 branches were processed

Finished examining sbikram/docker-ci-cd

01:24:02 2 repositories were processed
[Thu Dec 07 01:24:02 GMT 2017] Finished organization scan. Scan took 7.1 sec
Finished: SUCCESS

I committed some changes to code and build it again

dev:docker-ci-cd bikramsingh$ git add .
dev:docker-ci-cd bikramsingh$ git commit -m "update"
[master 9b91680] update
 1 file changed, 4 insertions(+), 4 deletions(-)
dev:docker-ci-cd bikramsingh$ git push -u origin master
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 321 bytes | 321.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/sbikram/docker-ci-cd.git
 25f2e15..9b91680 master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.

Build can be automatically trigger on Jenkins as soon as developer commits the code on Github using Webhooks.

Review the code and Jenkinsfile at https://github.com/sbikram/docker-ci-cd

Ref : https://github.com/krasi-georgiev

Thank you.