How to train your Jenkins?
Download as PPTX, PDF1 like633 views
The document discusses continuous integration (CI) practices at eBay and PayPal. It describes their previous model of having thousands of individual Jenkins instances running on separate VMs, which led to poor resource utilization and high costs. Their new architecture leverages Apache Mesos and containers to run lightweight Jenkins masters and slaves across a cluster, improving efficiency while maintaining user experience. Key contributions to the Jenkins Mesos plugin from eBay are also outlined.
How to train your Jenkins?
- 1. Jenkins User Conference San Francisco #jenkinsconf How to train your Jenkins? … to Use Your Resources Effectively and Get More Value for Each Buck Spent on Resources Manivannan Selvaraj & Soma Oruganti eBay Inc. ww.ebay.com October 23, 2014 #jenkinsconf
- 2. Jenkins User Conference San Francisco #jenkinsconf At eBay Inc. we take CI very seriously
- 3. Jenkins User Conference San Francisco #jenkinsconf Background - CI in PayPal • One Ginormous Jenkins Instance, customized for PayPal • 40,000 runnable jobs • 12,000 builds are run each day • Java, Node, Python, and C++ stacks, Fortify and Freestyle jobs too • Unit test and deploy flows • One Jenkins master with a pool of a hundred Jenkins slaves • Specialized “Monster Machine” stages build 40,000,000 lines of code in a half hour
- 4. Jenkins User Conference San Francisco #jenkinsconf Background - CI in eBay Marketplaces Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e Jenkins CI Wo rks pac e • Individual VMs running Jenkins for each Application • 2500+ VMs in use Wo rks pac e • Each Jenkins would have 2 executor on master to perform builds
- 5. Jenkins User Conference San Francisco #jenkinsconf Individual Jenkins VMs Pros Can add their own plug-ins Can upgrade to various versions of Jenkins Issues are isolated to one instance Cons Poor resource utilization - < 10% Cost of maintaining 2500+ VMs – Millions of $ every year Provisioning VMs takes several minutes Individual instances have local storage of limited size CI’s can disappear, losing all data and build history
- 6. Jenkins User Conference San Francisco #jenkinsconf New CI Architecture - Challenges • Put a smile on Dev-Ops • Optimal utilization of resources • Preserve user experience • High availability • Self healing infrastructure
- 7. Jenkins User Conference San Francisco #jenkinsconf Light weight Jenkins Apache Mesos Cloud … Light weight Jenkins for each user Developer 1 Developer 2 Developer n 0.4 CPU 512 MB mem
- 8. Jenkins User Conference San Francisco #jenkinsconf Mesos Architecture
- 9. Jenkins User Conference San Francisco #jenkinsconf Build slaves - Jenkins Mesos Plugin • Jenkins Mesos plugin community is very active • We have contributed enhancements/patches to the plugin
- 10. Jenkins User Conference San Francisco #jenkinsconf CI Application life cycle CIAAS Marathon Mesos Master Mesos Cluster NFS Request for CI Zookeeper Jenkins Master Jenkins Slave
- 11. Jenkins User Conference San Francisco #jenkinsconf Jenkins Mesos Plugin - our Contributions • Custom workspace • Multiple flavors of slaves • Access to workspace • Enable Maven jobs • Disconnect of slaves after idle timeout
- 12. Jenkins User Conference San Francisco #jenkinsconf
- 13. Jenkins User Conference San Francisco #jenkinsconf Thank You To Our Sponsors Platinum Gold Silver Corporate