Scaling your jenkins master with docker
Download as ODP, PDF5 likes1,221 views
This document discusses scaling a Jenkins master using Docker. It describes how the company Wyplay was using a single Jenkins master to manage over 500 jobs, which led to performance and reliability issues. It then explains how Wyplay set up multiple Docker containers running Jenkins masters to address these issues. It provides details on installing and configuring Jenkins in Docker containers, managing plugins, backups, and monitoring across multiple containerized masters. Finally, it discusses future directions like using orchestration tools and programmatically configuring Jenkins systems.
Scaling your jenkins master with docker
- 1. Scaling Your Jenkins Master with Docker London, UK 23-24 June
- 2. #jenkinsconf Contents ● WHO: Wyplay ● WHAT: Docker, Docker in Jenkins, Jenkins in Docker ● WHY: Situation Before, Issues ● HOW: Setup, Usage ● WHERE: Future 2
- 3. #jenkinsconf Contents ● WHO: Wyplay ● WHAT: Docker, Docker in Jenkins, Jenkins in Docker ● WHY: Situation Before, Issues ● HOW: Setup, Usage ● WHERE: Future 3
- 4. #jenkinsconf WHO am I I Worked in Queueing Networks, Simulation, Smartcards, now TV Software for Operators – Develop Set-top-box OS (Canal+, Proximus, SkyIt,..) – 80% of Developers 4
- 5. #jenkinsconf Contents ● WHO: Wyplay ● WHAT: Docker, Docker in Jenkins, Jenkins in Docker ● WHY: Situation Before, Issues ● HOW: Setup, Usage ● WHERE: Future 5
- 6. #jenkinsconf WHAT: Docker ● Company (Docker Inc) ● Open Source (Apache license) platform for building, shipping, running distributed applications ● Technology: manage and run lightweight Linux containers – Isolation – Portability, Shipping 6
- 7. #jenkinsconf WHAT: Docker in Jenkins 7 ● Docker Plugin – Dynamically provision slaves – Run build – Tear-down slave ● Slaves in the Cloud
- 8. #jenkinsconf WHAT: Jenkins in Docker ● Official Jenkins LTS docker image – From Cloudbees, sponsor of DockerCon – Open Source https://github.com/jenkinsci/docker – Easy to use docker pull jenkins docker run -p 8080:8080 jenkins 8
- 9. #jenkinsconf Contents ● WHO: Wyplay ● WHAT: Docker, Docker in Jenkins, Jenkins in Docker ● WHY: Situation Before, Issues ● HOW: Setup, Usage ● WHERE: Future 9
- 10. #jenkinsconf WHY: Situation Before 10 ● One Master – 34 attached slaves – 527 jobs – 78 plugins – admin users ● Fast-paced environment. No real control
- 11. #jenkinsconf WHY: Issues 11 ● Performance ● Reliability ● Inability to upgrade ● Downtime => Projects spin-off masters ● Security
- 12. #jenkinsconf Contents ● WHO: Wyplay ● WHAT: Docker, Docker in Jenkins, Jenkins in Docker ● WHY: Situation Before, Issues ● HOW: Setup, Usage ● WHERE: Future 12
- 13. #jenkinsconf HOW: Setup for one master (1) ● Three possibilities for /var/jenkins_home – inside the container: docker run -p 8080:8080 jenkins – mounted from a host directory: docker run -v <host-path>:/var/jenkins_home jenkins – mounted from a data volume container: docker run -v /var/jenkins_home --name=data busybox true docker run --volumes-from=data jenkins 13
- 14. #jenkinsconf HOW: Setup for one master (2) ● Summary for one master: – pull the official Jenkins image: $ docker pull jenkins – create the data volume and master containers: $ docker run -v /var/jenkins_home --name=frog-data busybox true – run the master container: $ docker run -t -i -u root -p 8084:8080 -p 50004:50004 --volumes-from=frog-data --name=frog jenkins 14
- 15. #jenkinsconf HOW: Setup n masters ● Launch an httpd dispatcher (e.g., Apache) – 443 (https) → vhosts(name) → 808i ● Launch docker containers: – the Data Volume containers, with names – the Jenkins masters containers linked with --volumes- from and 808i / 5000i ports – map each container port 8080 to host ports 8080+i 15
- 17. #jenkinsconf HOW: Admin Usage (1) ● Install plugins (from /tmp/plugins) $ docker run -t -i --rm --volumes-from=frog-data -v /tmp:/tmp ubuntu bash root@1a210c0a1846:/# cd /var/jenkins_home/plugins root@1a210c0a1846:/# cp /tmp/plugins/* . root@1a210c0a1846:/# exit ● Similar technique for: – full backup – debug data (SCM Sync git repos, Plots csv, etc.) 17
- 18. #jenkinsconf HOW: Admin Usage (2) ● Change image – stop container, destroy, create a new one $ docker stop frog $ docker rm frog $ docker run … --volumes-from=frog-data --name frog $ docker logs -f frog 18
- 19. #jenkinsconf HOW: Admin Usage (3) ● Three levels of backup – Vmware: dump of entire docker server – Docker: copy all Data Volume – Jenkins: SCM Sync plugin ● Three levels of monitoring – Vmware, Docker, Jenkins ● Security: Role Strategy plugin 19
- 20. #jenkinsconf Contents ● WHO: Wyplay ● WHAT: Docker, Docker in Jenkins, Jenkins in Docker ● WHY: Situation Before, Issues ● HOW: Setup, Usage ● WHERE: Future 20
- 21. #jenkinsconf WHERE: do we go from here? ● Orchestration – manual setup ok for a few – more => orchestration tool (e.g., Docker Compose) 21
- 22. #jenkinsconf WHERE: do we go from here? 22 ● Docker vs Config/Automation tools (chef, puppet, ansible, saltstack) Before: use ansible to setup hardware/VM, install packages, deploy code, run app. After: use ansible to setup hardware/VM, install Docker, run containers. use Dockerfiles to install packages, deploy code, run app.
- 23. #jenkinsconf WHERE: do we go from here? ● Monitoring (docker level) ● Find a way to “Configure System” programmatically – LDAP – Plugins upgrades – TCP port for slaves – Markup formatter – SCM Sync, Roles, IRC, etc. 23
- 24. #jenkinsconf Thank You @cmuller13 on twitter 24