JUC Europe 2015: Using Infrastructure Nodes Wisely With Jenkins and Apache Mesos
1 like283 views
The document presents a session at the Jenkins Conference detailing the effective use of infrastructure nodes with Jenkins and Apache Mesos. It covers topics including static and dynamic partitioning, the benefits and architecture of Mesos, and the implementation of a Jenkins Mesos plugin. The presentation also includes demonstrations and discusses challenges and strategies for integrating Jenkins with Mesos clusters.
![#jenkinsconf
Slave Resources and Attributes
Key-Value pairs (characteristics of slave)
Predefined resources
cpu, memory, disks, ports
Custom attributes
os, rack, etc.
Can be passed to slave while running them via –attributes/--resources
commandline option (separated by a semi-colon)
e.g. --attribute='os:centos'
--resources='cpus:24;mem:24576;disk:409600;ports:[21000-24000]
26](https://image.slidesharecdn.com/juc-2015-europe-infrastructure-nodes-bhattacharya-160623160645/85/JUC-Europe-2015-Using-Infrastructure-Nodes-Wisely-With-Jenkins-and-Apache-Mesos-26-320.jpg)
JUC Europe 2015: Using Infrastructure Nodes Wisely With Jenkins and Apache Mesos
- 1. #jenkinsconf Using Infrastructure Nodes Wisely With Jenkins and Apache Mesos London, 23 & 24 June 2015 1
- 2. #jenkinsconf whoami Pradeepto Bhattacharya / @pradeepto 2
- 3. #jenkinsconf Agenda Your Current Build / Test Cluster ( and its problems ) Static Partitioning Scaling and Limitations Your Future Build / Test Cluster Using Mesos Dynamic Partitioning Benefits Concepts Jenkins Mesos Plugin Demo 3
- 4. #jenkinsconf #jenkinsconf The Story So Far … (or our problem statement) 4
- 5. #jenkinsconf Your Usual (Build/Test) Cluster 5
- 6. #jenkinsconf Scaling and Limitations Machines You have have only so much machines! Money You have only so much money! 6
- 8. #jenkinsconf Apache Mesos Very brief history Omega and Borg Benjamin Hindman and merry men! Production grade Who uses it right now? http://mesos.apache.org/documentation/latest/powered-by-mesos/ 8
- 10. #jenkinsconf An Operating System OS for your cluster! 10
- 12. #jenkinsconf Your Old Cluster (but with a difference) 12
- 14. #jenkinsconf Benefits ROI Better Utilization Isolation (using cgroups on the top of HDFS) Efficiency 14
- 16. #jenkinsconf Architechture Image Credit : http://mesos.apache.org/documentation/latest/mesos-architecture/ 16
- 17. #jenkinsconf Frameworks Mesos Applications are called Frameworks Runs on the top of Mesos Consists of two components Schedulers Executors 17
- 19. #jenkinsconf August Club of Frameworks 19
- 20. #jenkinsconf Frameworks Periodic Table Image Credit : http://open.mesosphere.com/frameworks/ 20
- 26. #jenkinsconf Slave Resources and Attributes Key-Value pairs (characteristics of slave) Predefined resources cpu, memory, disks, ports Custom attributes os, rack, etc. Can be passed to slave while running them via –attributes/--resources commandline option (separated by a semi-colon) e.g. --attribute='os:centos' --resources='cpus:24;mem:24576;disk:409600;ports:[21000-24000] 26
- 27. #jenkinsconf Strategies Jenkins Master running stand-alone Jenkins Master running on Mesos Cluster via Marathon 27
- 28. #jenkinsconf Gotchas! LIBPROCESS_IP # LIBPROCESS_IP=10.10.20.100 <process_trigger_command> Firewall and blocked ports Dirty slaves 28
- 30. #jenkinsconf Thanks CloudBees and organisers of JUC Europe 2015 Contributors of Jenkins, Mesos, Mesos plugin My Family Kalpak, Shikha, Lenin Girija and Rupali (my partners in CI related crimes) 30
- 31. #jenkinsconf Manning Mesos In Action Coupon Code : mlignazio 50% off Mesos in Action, all formats, valid through June 3031
- 33. #jenkinsconf Please Share Your Feedback • Did you find this session valuable? • Please share your thoughts in the • Jenkins User Conference Mobile App. • Find the session in the app and click • on the feedback area. 33