SlideShare a Scribd company logo
CI for ALL
Agenda
● CI - Why, What & How
● YAML file and JJB
● CI Work-flow
● Build triggers
● Parameterize the jobs
● Build jobs using JJB
● Q&A session
Why CI?
● Company Goal
● Delay in response
● Env issues & User errors
● Inefficient use of tools/resources
● Duplicate test reports
● Compromise in Quality
What does CI do?
● Automatic execution of tests
● Integrate different tools
● Latest & greatest pkgs
● Clean env
● TIER based runs
● Centralized reporting
How to setup CI?
● Jenkins Master/Slave
– Install/Config/Plugin
● Requirements and project details
– YAML file
– JJB – Jenkins Job Builder
● Triggers
● Repo creation
● Provision & teardown
● Runtest
● YAML file and JJB
YAML file
Global defaults
Project vars & Jobs
Jenkins vars
Json file
Restraint xml
Builders
Parameters
Jobs
Jenkins Jobs
Trigger
Repo creation
Provision
Runtest
Teardown
Build jobs from Jenkins
JJB(Python script)
GIT repo
Jenkins
Master
Config
file(Creds)
● CI work-flow
Brew
Trigger
Provision
Resources
Runtest
Teardown
Resources
Brew build
Redhat CI
PluginYAML file
Mail
Notifications
IRC
Notifications
JMS
Selector
Brew
bus
Jenkins Jobs
JJB
Project variables
● jobs:
- '{prefix}-{testsuite}-{arch}-provision'
- '{prefix}-{testsuite}-{arch}-runtest'
- '{prefix}-{testsuite}-{arch}-teardown'
● testsuite:
- test1
- test2
- test3
● Prefix: rhds
● brew-tag: rhel-7.2-candidate
● arch: x86_64
Variables contd...
● Replace variable with values:
– Variable within YAML file
● '{release}'
● '{prefix}-{testsuite}-{arch}-provision'
– Parameters
● $PATCH_ID
● $BUILD_ID
– Jenkins env variables
● $WORKSPACE
● $JENKINS_HOME
Build triggers
● Upstream job - Trigger
● Downstream – Provision
● Triggers:
- ci-trigger:
jms-selector: |
CI_TYPE IN ('brew-tag')
AND
CI_NAME IN ('389-ds-base','nss','openldap')
AND
tag IN '{brew-tag}'
Build triggers contd...
● Periodic build trigger
● Jenkins job trigger
– Build when jobs built
– Build when jobs promoted
– Build based on results of multiple jobs
● Script trigger – Shell script
● Gerrit trigger
– Gerrit server details
– When patchset created, merged
Provision
● Beaker/Openstack
– JSON File
– provision_resources.sh
● Upstream job - Trigger
● Downstream job - Runtest
● Copy artifcats
– Inject information to RESOURCES.txt
Provision contd...
- builder:
name: 'provision-resources'
builders:
- shell: |
#!/bin/bash
$WORKSPACE/ci-ops-
central/bootstrap/provision_resources.sh --site=$SITE 
--project_defaults=dirsrv-tests/config/project_defaults 
--topology={topology}-$PROVISIONER 
--ssh_keyfile=dirsrv-tests/config/keys/idm-jenkins 
--name={testsuite}
TR_STATUS=$?
JSON File
"family": "RedHatEnterpriseLinux7",
“arch": "x86_64",
"tag": ["RTT_ACCEPTED"],
"variant": "Server",
"hostrequire": ["arch=x86_64"]
"name": "ci-ops-rhel-7-1",
"count": "1",
"flavor": "m1.medium",
"image": "RHEL-7.2-Server-x86_64-latest",
"ip": "10.8.60.20",
"name": "nocp9.idm.lab.eng.rdu2.redhat.com",
"private_ip": "10.8.60.20”
Runtest
● Restraint xml file
– Beaker task
– Install, configure and execute tests
● Ansible
– Ansible playbook
– Install pkgs
– Run your tests
● Upstream – provision
– Bkr job ID, IP address & Hostname
● Downstream - teardown
Config file
[jenkins]
user=idm-jenkins
password=Secret123
url=https://idm-qe-jenkins.rhev-ci-
vms.eng.rdu2.redhat.com
ignore_cache=False
● Jenkins Job Builder
builders:
- shell: |
JENKINS_URL=${JENKINS_URL:-"https://localhost/"}
pushd $WORKSPACE/dirsrv-tests
Popd
cd $WORKSPACE
python -m jenkins_jobs.cmd --conf $WORKSPACE/
$CONIG_FILE --flush-cache update $WORKSPACE/
$GIT_REPO/$CI_DIR
Example for RHDS:
python -m jenkins_jobs.cmd --conf
$WORKSPACE/dirsrv-tests/jobs/config --flush-cache update
$WORKSPACE/dirsrv-tests/jobs/rhel72
Parameterize jobs
● Change variable values
– Troubleshooting & Manual execution
● choice:
name: tag
choices:
- rhel-7.2-z-candidate
- rhel-7.2-candidate
description: "Brew tag for this build."
● string:
name: REPORT_EMAIL
default: redhat-list@redhat.com
description: Default email id for reports
JJB with Gerrit refspec
● Add a parameter - GERRIT_REFSPEC
● Add a build step – Execute shell
pushd $WORKSPACE/dirsrv-tests
git pull
https://code.engineering.redhat.com/gerrit/dirsrv-tests
$GERRIT_REFSPEC
popd
● cd $WORKSPACE
● python -m jenkins_jobs.cmd --conf
$WORKSPACE/dirsrv-tests/jobs/config --flush-cache
update $WORKSPACE/dirsrv-tests/jobs/
$GIT_BRANCH
References
About CI in MOJO - https://mojo.redhat.com/search.jspa?q=CI
%20workflow
Setup CI with Jenkins -
http://www.vogella.com/tutorials/Jenkins/article.html
Jenkins CI plugin - https://mojo.redhat.com/docs/DOC-986839
CI Reference guide - https://mojo.redhat.com/docs/DOC-1030059
Questions

More Related Content

PDF
Docker in Action
PDF
Building with Firebase
PDF
Forget Grunt and Gulp! Webpack and NPM rule them all!
PDF
TDC2016SP - Esqueça Grunt ou Gulp. Webpack and NPM rule them all!
PDF
How we maintain 200+ Drupal sites in Georgetown University
PDF
GCPUG meetup 201610 - Dataflow Introduction
PDF
Using Backbone.js with Drupal 7 and 8
PDF
Devfest 2021' - Artifact Registry Introduction (Taipei)
Docker in Action
Building with Firebase
Forget Grunt and Gulp! Webpack and NPM rule them all!
TDC2016SP - Esqueça Grunt ou Gulp. Webpack and NPM rule them all!
How we maintain 200+ Drupal sites in Georgetown University
GCPUG meetup 201610 - Dataflow Introduction
Using Backbone.js with Drupal 7 and 8
Devfest 2021' - Artifact Registry Introduction (Taipei)

What's hot (20)

PDF
Mastering Grails 3 Plugins - GR8Conf EU 2016
PDF
以 Ktor 快速打造 Web 應用
PDF
Create a Varnish cluster in Kubernetes for Drupal caching - DrupalCon North A...
PDF
Environmental variables
PDF
[Kotlin Serverless 工作坊] 單元 3 - 實作 JSON API
PPTX
Orchestration? You Don't Need Orchestration. What You Want is Choreography.
PDF
Service Worker Presentation
PDF
以 Laravel 經驗開發 Hyperf 應用
PDF
用 Kotlin 做自動化工具
PDF
運用 Exposed 管理及操作資料庫
PPTX
NLIT 2011: Chef & Capistrano
PPTX
Cut your hair and get an azure webjob
PDF
Infrastructure Management in GCP
PDF
DevOps tools for everyone - Vagrant, Puppet and Webmin
PPTX
What Makes a Good Chef Cookbook? (May 2014 Edition)
PDF
Agiles Peru 2019 - Infrastructure As Code
PDF
[HKOSCon 2020] Build an api service using ktor rapidly
PPTX
Dropwizard Internals
PDF
Ansible 101 - Presentation at Ansible STL Meetup
PDF
以 Kotlin 快速打造 Mobile Backend
Mastering Grails 3 Plugins - GR8Conf EU 2016
以 Ktor 快速打造 Web 應用
Create a Varnish cluster in Kubernetes for Drupal caching - DrupalCon North A...
Environmental variables
[Kotlin Serverless 工作坊] 單元 3 - 實作 JSON API
Orchestration? You Don't Need Orchestration. What You Want is Choreography.
Service Worker Presentation
以 Laravel 經驗開發 Hyperf 應用
用 Kotlin 做自動化工具
運用 Exposed 管理及操作資料庫
NLIT 2011: Chef & Capistrano
Cut your hair and get an azure webjob
Infrastructure Management in GCP
DevOps tools for everyone - Vagrant, Puppet and Webmin
What Makes a Good Chef Cookbook? (May 2014 Edition)
Agiles Peru 2019 - Infrastructure As Code
[HKOSCon 2020] Build an api service using ktor rapidly
Dropwizard Internals
Ansible 101 - Presentation at Ansible STL Meetup
以 Kotlin 快速打造 Mobile Backend
Ad

Viewers also liked (20)

DOC
Persuasive speeches in history
PPTX
Trabajo práctico
DOC
Ensayo final
PDF
Conexión remota-a-oracle-database
PPT
Оскари "Глисо"
PPTX
The Denial of Alfred Wegener's Continental Drift. Allan Krill, NTNU
PPSX
presentaciones e imagenes
PPTX
портфоліо
PDF
8. otsingumootorid
DOCX
AA-ATS
PPTX
Tech Interview Job Prep
PPTX
Investigacion invertebrados
ODP
Aspect-Oriented Programming for PHP
DOCX
Analisis Estructural I - Metodo de la Viga Conjugada
PPTX
Using Text Embeddings for Information Retrieval
PDF
Transparencias dinamica orden
DOC
PPTX
DOCX
HOANG MINH DOAN
PPTX
IoT Analytics from Edge to Cloud - using IBM Informix
Persuasive speeches in history
Trabajo práctico
Ensayo final
Conexión remota-a-oracle-database
Оскари "Глисо"
The Denial of Alfred Wegener's Continental Drift. Allan Krill, NTNU
presentaciones e imagenes
портфоліо
8. otsingumootorid
AA-ATS
Tech Interview Job Prep
Investigacion invertebrados
Aspect-Oriented Programming for PHP
Analisis Estructural I - Metodo de la Viga Conjugada
Using Text Embeddings for Information Retrieval
Transparencias dinamica orden
HOANG MINH DOAN
IoT Analytics from Edge to Cloud - using IBM Informix
Ad

Similar to Ci for all (20)

PPTX
Make BDD great again
PPTX
Iac d.damyanov 4.pptx
PDF
Automating complex infrastructures with Puppet
PDF
Practical Chef and Capistrano for Your Rails App
PDF
Grunt.js and Yeoman, Continous Integration
PDF
Declarative Infrastructure Tools
PDF
Automating Complex Setups with Puppet
PDF
Yaetos Tech Overview
PPTX
Automating Software Development Life Cycle - A DevOps Approach
PDF
Yaetos_Meetup_SparkBCN_v1.pdf
PPT
Tp web
PPT
一种多屏时代的通用 web 应用架构
PPT
nodejs tutorial foor free download from academia
PDF
The Accidental DBA
PDF
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
PPTX
PPT
Build Automation of PHP Applications
PPTX
FP - Découverte de Play Framework Scala
PDF
Implementing a build manager in Ada
PDF
Grails 101
Make BDD great again
Iac d.damyanov 4.pptx
Automating complex infrastructures with Puppet
Practical Chef and Capistrano for Your Rails App
Grunt.js and Yeoman, Continous Integration
Declarative Infrastructure Tools
Automating Complex Setups with Puppet
Yaetos Tech Overview
Automating Software Development Life Cycle - A DevOps Approach
Yaetos_Meetup_SparkBCN_v1.pdf
Tp web
一种多屏时代的通用 web 应用架构
nodejs tutorial foor free download from academia
The Accidental DBA
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
Build Automation of PHP Applications
FP - Découverte de Play Framework Scala
Implementing a build manager in Ada
Grails 101

Recently uploaded (20)

PPTX
Sustainable Forest Management ..SFM.pptx
PDF
IKS PPT.....................................
PPTX
HOW TO HANDLE THE STAGE FOR ACADEMIA AND OTHERS.pptx
PPTX
Kompem Part Untuk MK Komunikasi Pembangunan 5.pptx
PPTX
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
PPTX
PurpoaiveCommunication for students 02.pptx
PDF
_Nature and dynamics of communities and community development .pdf
PPTX
Anesthesia and it's stage with mnemonic and images
PDF
6.-propertise of noble gases, uses and isolation in noble gases
PPTX
Lesson-7-Gas. -Exchange_074636.pptx
PDF
PM Narendra Modi's speech from Red Fort on 79th Independence Day.pdf
PDF
Presentation1 [Autosaved].pdf diagnosiss
PDF
Microsoft-365-Administrator-s-Guide_.pdf
PPTX
Research Process - Research Methods course
PPTX
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
DOCX
Action plan to easily understanding okey
PPTX
Shizophrnia ppt for clinical psychology students of AS
PDF
public speaking for kids in India - LearnifyU
PDF
Yusen Logistics Group Sustainability Report 2024.pdf
PPTX
chapter8-180915055454bycuufucdghrwtrt.pptx
Sustainable Forest Management ..SFM.pptx
IKS PPT.....................................
HOW TO HANDLE THE STAGE FOR ACADEMIA AND OTHERS.pptx
Kompem Part Untuk MK Komunikasi Pembangunan 5.pptx
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
PurpoaiveCommunication for students 02.pptx
_Nature and dynamics of communities and community development .pdf
Anesthesia and it's stage with mnemonic and images
6.-propertise of noble gases, uses and isolation in noble gases
Lesson-7-Gas. -Exchange_074636.pptx
PM Narendra Modi's speech from Red Fort on 79th Independence Day.pdf
Presentation1 [Autosaved].pdf diagnosiss
Microsoft-365-Administrator-s-Guide_.pdf
Research Process - Research Methods course
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
Action plan to easily understanding okey
Shizophrnia ppt for clinical psychology students of AS
public speaking for kids in India - LearnifyU
Yusen Logistics Group Sustainability Report 2024.pdf
chapter8-180915055454bycuufucdghrwtrt.pptx

Ci for all

  • 2. Agenda ● CI - Why, What & How ● YAML file and JJB ● CI Work-flow ● Build triggers ● Parameterize the jobs ● Build jobs using JJB ● Q&A session
  • 3. Why CI? ● Company Goal ● Delay in response ● Env issues & User errors ● Inefficient use of tools/resources ● Duplicate test reports ● Compromise in Quality
  • 4. What does CI do? ● Automatic execution of tests ● Integrate different tools ● Latest & greatest pkgs ● Clean env ● TIER based runs ● Centralized reporting
  • 5. How to setup CI? ● Jenkins Master/Slave – Install/Config/Plugin ● Requirements and project details – YAML file – JJB – Jenkins Job Builder ● Triggers ● Repo creation ● Provision & teardown ● Runtest
  • 6. ● YAML file and JJB YAML file Global defaults Project vars & Jobs Jenkins vars Json file Restraint xml Builders Parameters Jobs Jenkins Jobs Trigger Repo creation Provision Runtest Teardown Build jobs from Jenkins JJB(Python script) GIT repo Jenkins Master Config file(Creds)
  • 7. ● CI work-flow Brew Trigger Provision Resources Runtest Teardown Resources Brew build Redhat CI PluginYAML file Mail Notifications IRC Notifications JMS Selector Brew bus Jenkins Jobs JJB
  • 8. Project variables ● jobs: - '{prefix}-{testsuite}-{arch}-provision' - '{prefix}-{testsuite}-{arch}-runtest' - '{prefix}-{testsuite}-{arch}-teardown' ● testsuite: - test1 - test2 - test3 ● Prefix: rhds ● brew-tag: rhel-7.2-candidate ● arch: x86_64
  • 9. Variables contd... ● Replace variable with values: – Variable within YAML file ● '{release}' ● '{prefix}-{testsuite}-{arch}-provision' – Parameters ● $PATCH_ID ● $BUILD_ID – Jenkins env variables ● $WORKSPACE ● $JENKINS_HOME
  • 10. Build triggers ● Upstream job - Trigger ● Downstream – Provision ● Triggers: - ci-trigger: jms-selector: | CI_TYPE IN ('brew-tag') AND CI_NAME IN ('389-ds-base','nss','openldap') AND tag IN '{brew-tag}'
  • 11. Build triggers contd... ● Periodic build trigger ● Jenkins job trigger – Build when jobs built – Build when jobs promoted – Build based on results of multiple jobs ● Script trigger – Shell script ● Gerrit trigger – Gerrit server details – When patchset created, merged
  • 12. Provision ● Beaker/Openstack – JSON File – provision_resources.sh ● Upstream job - Trigger ● Downstream job - Runtest ● Copy artifcats – Inject information to RESOURCES.txt
  • 13. Provision contd... - builder: name: 'provision-resources' builders: - shell: | #!/bin/bash $WORKSPACE/ci-ops- central/bootstrap/provision_resources.sh --site=$SITE --project_defaults=dirsrv-tests/config/project_defaults --topology={topology}-$PROVISIONER --ssh_keyfile=dirsrv-tests/config/keys/idm-jenkins --name={testsuite} TR_STATUS=$?
  • 14. JSON File "family": "RedHatEnterpriseLinux7", “arch": "x86_64", "tag": ["RTT_ACCEPTED"], "variant": "Server", "hostrequire": ["arch=x86_64"] "name": "ci-ops-rhel-7-1", "count": "1", "flavor": "m1.medium", "image": "RHEL-7.2-Server-x86_64-latest", "ip": "10.8.60.20", "name": "nocp9.idm.lab.eng.rdu2.redhat.com", "private_ip": "10.8.60.20”
  • 15. Runtest ● Restraint xml file – Beaker task – Install, configure and execute tests ● Ansible – Ansible playbook – Install pkgs – Run your tests ● Upstream – provision – Bkr job ID, IP address & Hostname ● Downstream - teardown
  • 17. ● Jenkins Job Builder builders: - shell: | JENKINS_URL=${JENKINS_URL:-"https://localhost/"} pushd $WORKSPACE/dirsrv-tests Popd cd $WORKSPACE python -m jenkins_jobs.cmd --conf $WORKSPACE/ $CONIG_FILE --flush-cache update $WORKSPACE/ $GIT_REPO/$CI_DIR Example for RHDS: python -m jenkins_jobs.cmd --conf $WORKSPACE/dirsrv-tests/jobs/config --flush-cache update $WORKSPACE/dirsrv-tests/jobs/rhel72
  • 18. Parameterize jobs ● Change variable values – Troubleshooting & Manual execution ● choice: name: tag choices: - rhel-7.2-z-candidate - rhel-7.2-candidate description: "Brew tag for this build." ● string: name: REPORT_EMAIL default: redhat-list@redhat.com description: Default email id for reports
  • 19. JJB with Gerrit refspec ● Add a parameter - GERRIT_REFSPEC ● Add a build step – Execute shell pushd $WORKSPACE/dirsrv-tests git pull https://code.engineering.redhat.com/gerrit/dirsrv-tests $GERRIT_REFSPEC popd ● cd $WORKSPACE ● python -m jenkins_jobs.cmd --conf $WORKSPACE/dirsrv-tests/jobs/config --flush-cache update $WORKSPACE/dirsrv-tests/jobs/ $GIT_BRANCH
  • 20. References About CI in MOJO - https://mojo.redhat.com/search.jspa?q=CI %20workflow Setup CI with Jenkins - http://www.vogella.com/tutorials/Jenkins/article.html Jenkins CI plugin - https://mojo.redhat.com/docs/DOC-986839 CI Reference guide - https://mojo.redhat.com/docs/DOC-1030059