Integrate Jenkins with Github

 GIT and Jenkins

Install Git plugin and GitHub plugin on Jenkins, in my case the jenkins login url is http://localhost:8080

Click on Manage Jenkins -> Manage Plugins -> search for Git plugin and GitHub plugin and install it

Integrate Jenkins with Github:

Login to GitHub and get the Project and Repository URL. In my case

Project URL: https://github.com/yashwanthsn2020/automation/

Repository URL: https://github.com/yashwanthsn2020/automation.git

Create a new job in jenkins -> jenkins-git-integration -> Freestyle project -> click OK

On the General tab -> choose GitHub project and add the Project URL

Under Source Code Management -> choose Git and add Repository URL

Branches to build: by default it will be master branch, you can add other branches 

Under Build Triggers section I chose Build Periodically since my jenkins run on localhost. 

I.e, http://localhost:8080, we can not add a webhook from GitHub to jenkins for a localhost

If Jenkins is not on localhost, you can choose "GitHub hook trigger for GITScm polling"

Click on Save and -> Back to Dashboard -> choose the job -> Jenkins-git-integration -> click on Build Now

Here is the console output from Jenkins

If you choose GitHub hook trigger for GITScm polling under Build Triggers, then you have to login to GitHub -> Settings -> Webhooks -> Add webhook

Click on Add webhook

Clone Git Repository on Jenkins and execute a script

 GIT and Jenkins

Create a new job and select Freestyle project. Go directly to Build section and select "Execute Shell"

The jenkin job runs as SYSTEM user, so ensure you have access to folder where you clone the files.

In my case, I have a SYSTEM user on the system with $HOME directory /home/system. Will close git repo into scripts folder

added following entries in the command section

cd /home/system

rm -rf scripts

git clone https://github.com/yashwanthsn2020/automation.git scripts
./scripts/api_showpassword.sh

./scripts/listfiles.sh

./scripts/crontab.sh

 Click on Save

Click on Build Now, check the console output

Install Jenkins on Linux

 Jenkins

Add the following key on the linux

wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
Add following entries in the /etc/apt/sources.list file (use Vi editor)

deb https://pkg.jenkins.io/debian-stable binary/

Java is required to run jenkins. Refer to Java Requirements Page. We will install Java 11 version. Jenkins 2.1+ versions requires Java 8 or Java 11

Update package index

sudo apt-get update

Install Jenkins, run

sudo apt-get install jenkins

Once the installation is complete. launch Jenkins in URL. I have configured hostname devops along with localhost. The URLs are

http://localhost:8080

http://devops:8080

Enter the admin password from /var/lib/jenkins/secrets/initialAdminPassword. If copy doesn't work, type in manually

Click on Install suggested Plugins

Jenkins Instance is ready for the use

Jenkins Pipeline

Jenkins

Login to Jenkins instance. Create a pipeline script test-pipeline
Click on New Item -> Enter the Item Name -> Click on Pipeline -> Click ok

Scroll down to Pipeline section, write the pipeline script. For now we will create a script that consists of
1. Pipeline structure
2. Stage/Stages
3. Steps
Within the Pipeline structure, we will have Stage or Stages and within Stage or Stages we will have steps
A simple script would like this

Click on save and click on Build Now
Once the Build is complete, the console output will look like this

Lets add more steps, click on Configure and enter the following code
Which has three stages and each stage containing one step each

Click on Build Now, you see the following screenshot displaying each stage, with success or failure and log messages. Click on each green area and click on Logs

Check the console output, it will look like

Lets Add sh, build, checkout, git, input into the pipeline script
Using sh - we are executing commands
Using build - we are running 'echo-test' script
Using checkout and git to pull repository
Input is used to review, Proceed or Abort the pipeline build

pipeline {
    agent any
    stages {
        stage('check Docker version') {
            steps {
                sh '/usr/bin/docker --version'
            }
        }
        stage('check Linux version') {
            steps {
                sh 'uname -a'
            }
        }
        stage('running echo-test') {
           steps {
               build 'echo-test'
           }
        } 
        stage('pulling git repository') {
           steps {
               checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'd4cd4712-affc-4933-a18f-dfbb90338097', url: 'https://github.com/yashwanthsn2020/automation.git'], [credentialsId: 'd4cd4712-affc-4933-a18f-dfbb90338097', url: 'https://github.com/yashwanthsn2020/yashwanthsn.git']]])
           }
        }
        stage('git:automation') {
           steps {
               git credentialsId: 'd4cd4712-affc-4933-a18f-dfbb90338097', url: 'https://github.com/yashwanthsn2020/automation.git'
            }
        }
        stage('approve') {
           steps {
               input 'Approve'
            }
        }
        
        
    }
}

Click Save -> Click on Build now

The Console output looks like this

Success Console Output

Started by user yashwanthsn
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/version-pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (check Docker version)
[Pipeline] sh
+ /usr/bin/docker --version
Docker version 19.03.11, build 42e35e61f3
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (check Linux version)
[Pipeline] sh
+ uname -a
Linux yashwanthsn-VirtualBox 5.4.0-26-generic #30-Ubuntu SMP Mon Apr 20 16:58:30 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (running echo-test)
[Pipeline] build (Building echo-test)
Scheduling project: echo-test
Starting building: echo-test #18
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (pulling git repository)
[Pipeline] checkout
using credential d4cd4712-affc-4933-a18f-dfbb90338097
using credential d4cd4712-affc-4933-a18f-dfbb90338097
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from 2 remote Git repositories
 > git config remote.origin.url https://github.com/yashwanthsn2020/automation.git # timeout=10
Fetching upstream changes from https://github.com/yashwanthsn2020/automation.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 
 > git fetch --tags --force --progress -- https://github.com/yashwanthsn2020/automation.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin1.url https://github.com/yashwanthsn2020/yashwanthsn.git # timeout=10
Fetching upstream changes from https://github.com/yashwanthsn2020/yashwanthsn.git
using GIT_ASKPASS to set credentials 
 > git fetch --tags --force --progress -- https://github.com/yashwanthsn2020/yashwanthsn.git +refs/heads/*:refs/remotes/origin1/* # timeout=10
Seen branch in repository origin/20Jun20
Seen branch in repository origin/master
Seen branch in repository origin/mytest
Seen branch in repository origin/revert-5-test
Seen branch in repository origin/test
Seen branch in repository origin/yashwanthsn2020-workflow
Seen branch in repository origin1/20Jun20
Seen branch in repository origin1/master
Seen branch in repository origin1/mytest
Seen 9 remote branches
 > git show-ref --tags -d # timeout=10
Checking out Revision cd91b37fa672d01052a1ece9b475ed88ef0f55a0 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f cd91b37fa672d01052a1ece9b475ed88ef0f55a0 # timeout=10
Commit message: "Merge pull request #13 from yashwanthsn2020/test"
 > git rev-list --no-walk cd91b37fa672d01052a1ece9b475ed88ef0f55a0 # timeout=10

[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (git:automation)
[Pipeline] git
using credential d4cd4712-affc-4933-a18f-dfbb90338097
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/yashwanthsn2020/automation.git # timeout=10
Fetching upstream changes from https://github.com/yashwanthsn2020/automation.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 
 > git fetch --tags --force --progress -- https://github.com/yashwanthsn2020/automation.git +refs/heads/*:refs/remotes/origin/* # timeout=10

 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision cd91b37fa672d01052a1ece9b475ed88ef0f55a0 (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f cd91b37fa672d01052a1ece9b475ed88ef0f55a0 # timeout=10
 > git branch -a -v --no-abbrev # timeout=10
 > git branch -D master # timeout=10
 > git checkout -b master cd91b37fa672d01052a1ece9b475ed88ef0f55a0 # timeout=10
Commit message: "Merge pull request #13 from yashwanthsn2020/test"
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (approve)
[Pipeline] input
Approve
Proceed or Abort

Approved by yashwanthsn
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Check http://<jenkinsurl>:8080/job/parameterized-pipeline/pipeline-syntax/ to learn more on different Steps available

SSH Remote Hosts setup in Jenkins

Jenkins

Login to Jenkins instance: http://devops/8080
Click on Manage Jenkins -> Configure System -> scroll down to "SSH remote hosts" section

Enter the Hostname, default port is 22
Credentials -> Click on Add -> Jenkins ->

Enter ID, Description, Username and then select Private Key and choose Enter directly, click on Add

To get the private ssh keys, login to the Hostname you entered in the previous screen/step and create .ssh folder under $HOME and generate rsa keys under $HOME/.ssh directory

ssh-keygen -t rsa

choose the directory and just hit enter twice when asked for a passphrase

cat id_rsa.pub >> ~/.ssh/authorized_keys (on the host)

chmod 700 ~/.ssh

chmod 600 ~/.ssh/authorized_keys ~/.ssh/id_rsa

Then copy the Private SSH key (id_rsa contents) and paste it back in the Add Credentials page of Jenkins -> Click on Add -> Click on Save

Go to Jenkins home page -> Click on New Item -> Enter a Item name -> Freestyle project -> click ok

Go directly to Build section -> select Execute shell script on remote host using ssh
Select the SSH Site

Under the command section -> add the commands

Click on Save

Click on Build Now

View Console Output

Jenkins Node Pipelines

 Jenkins

To create a Node in the pipelines structure, use the below step
node('exec') {
// some block
}
Click on New Item -> Pipeline -> under Pipeline section add the below code. When the integer value is zero, clone automation.git

node {
    i = 0
    if (i == 0)
    {
        echo "Value of i is $i"
        git "https://github.com/yashwanthsn2020/automation.git"
    }

Build the pipeline

Console Output

Add the below Node step to clone two repos based on the integer value

node {
    i = 0
    if (i == 0)
    {
        echo "Value of i is $i"
        git "https://github.com/yashwanthsn2020/automation.git"
    }
    i+=1
    if (i == 1)
    {
        echo "Value of i is $i"
        git "https://github.com/yashwanthsn2020/yashwanthsn.git"
    }
}

Click on Build Now and check the Console output

Jenkins Node Pipelines Parameterized

Jenkins

Login to Jenkins -> Click on New Item -> choose Pipeline -> Click Ok
On the configure page, select -> This project is parameterized
Enter Name and Choices

Go to Pipeline section and add the following Node code

node {
    if ("${GitRepos}" == "automation")
    {
        git "https://github.com/yashwanthsn2020/automation.git"
    }
    if ("${GitRepos}" == "yashwanthsn")
    {
        git "https://github.com/yashwanthsn2020/yashwanthsn.git"
    }
}

Click on save
Click on Build with Parameters
Select automation and Click on Build, it should clone the automation.git

Now try to build with yashwanthsn choices parameter

Install Jenkins using Ansible

  

Ansible

Install Jenkins using Ansible Playbook. Create a Playbook called Jenkins.yml

cat jenkins.yml
- hosts: localhost
  become: yes
  become_method: sudo
  become_flags: -s
  become_user: root
  tasks:
   - name: Install Jenkins
     shell: |
      wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
      sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > \
      /etc/apt/sources.list.d/jenkins.list'
      sudo apt-get update
      yes | sudo apt-get install jenkins

ansible@ansible-VirtualBox:~/Desktop$ ansible-playbook jenkins.yml -v
Using /etc/ansible/ansible.cfg as config file

PLAY [localhost] ******************************************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Install Jenkins] ************************************************************************************************************************************************************************************
[WARNING]: Consider using the get_url or uri module rather than running 'wget'.  If you need to use command because get_url or uri is insufficient you can add 'warn: false' to this command task or set
'command_warnings=False' in ansible.cfg to get rid of this message.
changed: [localhost] => changed=true
  cmd: |-
    wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
    sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > \
    /etc/apt/sources.list.d/jenkins.list'
    sudo apt-get update
    yes | sudo apt-get install jenkins
  delta: '0:03:37.888632'
  end: '2021-04-25 16:17:45.946913'
  rc: 0
  start: '2021-04-25 16:14:08.058281'
  stderr: 'Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).'
  stderr_lines: <omitted>
  stdout: |-
    OK
    Hit:1 http://in.archive.ubuntu.com/ubuntu groovy InRelease
    Ign:2 https://pkg.jenkins.io/debian-stable binary/ InRelease
    Hit:3 https://pkg.jenkins.io/debian-stable binary/ Release
    Hit:5 http://security.ubuntu.com/ubuntu groovy-security InRelease
    Hit:6 http://in.archive.ubuntu.com/ubuntu groovy-updates InRelease
    Hit:7 http://in.archive.ubuntu.com/ubuntu groovy-backports InRelease
    Hit:8 https://download.docker.com/linux/ubuntu groovy InRelease
    Reading package lists...
    Reading package lists...
    Building dependency tree...
    Reading state information...
    The following additional packages will be installed:
      daemon net-tools
    The following NEW packages will be installed:
      daemon jenkins net-tools
    0 upgraded, 3 newly installed, 0 to remove and 251 not upgraded.
    Need to get 68.3 MB/68.6 MB of archives.
    After this operation, 72.1 MB of additional disk space will be used.
    Do you want to continue? [Y/n] Get:1 https://pkg.jenkins.io/debian-stable binary/ jenkins 2.277.3 [68.3 MB]
    Fetched 15.5 MB in 3min 5s (83.8 kB/s)
    Selecting previously unselected package daemon.
    (Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 271868 files and directories currently installed.)
    Preparing to unpack .../daemon_0.6.4-1build2_amd64.deb ...
    Unpacking daemon (0.6.4-1build2) ...
    Selecting previously unselected package net-tools.
    Preparing to unpack .../net-tools_1.60+git20180626.aebd88e-1ubuntu2_amd64.deb ...
    Unpacking net-tools (1.60+git20180626.aebd88e-1ubuntu2) ...
    Selecting previously unselected package jenkins.
    Preparing to unpack .../jenkins_2.277.3_all.deb ...
    Unpacking jenkins (2.277.3) ...
    Setting up net-tools (1.60+git20180626.aebd88e-1ubuntu2) ...
    Setting up daemon (0.6.4-1build2) ...
    Setting up jenkins (2.277.3) ...
    Processing triggers for man-db (2.9.3-2) ...
    Processing triggers for systemd (246.6-1ubuntu1) ...
  stdout_lines: <omitted>

PLAY RECAP ************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0  

Once the step is complete, follow the below link to continue with Jenkins installation ->

http://localhost:8080

https://devopsapt.blogspot.com/2020/06/install-jenkins-on-linux.html

Jenkins is setup: