MPL: modular pipeline library - Dynamic Talks Milwaukee 4/11/2019
Download as PPTX, PDF2 likes979 views
The document provides an overview of the MPL (Modular Pipeline Library), a Jenkins pipeline library designed to enhance development collaboration and CI/CD processes. It covers key concepts such as continuous integration and delivery, pipeline structure, and shared libraries, along with demos for setup and operation. Additionally, it includes details on its modular architecture, unit testing, and nested libraries for project implementation.
![Our Agenda
2
● MPL Overview and why would you need it
● What is CICD and why it is important?
● [Demo] Setup tools
● Explore Jenkins Pipelines and MPL Role
● [Demo] MPL Operation
● MPL Internals](https://image.slidesharecdn.com/mplmodularpipelinelibrary-190412184751/85/MPL-modular-pipeline-library-Dynamic-Talks-Milwaukee-4-11-2019-2-320.jpg)
MPL: modular pipeline library - Dynamic Talks Milwaukee 4/11/2019
- 2. Our Agenda 2 ● MPL Overview and why would you need it ● What is CICD and why it is important? ● [Demo] Setup tools ● Explore Jenkins Pipelines and MPL Role ● [Demo] MPL Operation ● MPL Internals
- 3. 3 ● It is a Jenkins Pipeline Library ● It is written in Groovy ● Modular and Extendable by Design ● Helps Build Development Collaboration and CICD ● Supports Unit Testing for Safer Development So what is MPL exactly?
- 4. What Is CI/CD? Code Review Build Deploy Functional Testing Business Testing Security Testing Perf. Testing Release 4 Continuous Integration / Continuous Delivery Principles
- 5. What Is CI/CD? Code Review Build Deploy Functional Testing Business Testing Security Testing Perf. Testing Release 5 Continuous Integration / Continuous Delivery Principles
- 6. What Is CI/CD? Code Review Build Deploy Functional Testing Business Testing Security Testing Perf. Testing Release 6 Pipeline Continuous Integration / Continuous Delivery Principles
- 7. Demo : Setup Tools 7 * (BlueOcean version)
- 8. Demo : Setup Tools 8 # Centos 7; Open HTTP Traffic; All API permissions ● # Make sure we have docker until yum install -y docker; do sleep 5; done ● # Start docker service systemctl start docker ● # Run official blueocean jenkins image docker run -d -p 80:8080 jenkinsci/blueocean ● # Make sure Jenkins is up CONTAINER=$(docker ps | tail -n 1| cut -f 1 -d ' ') until docker logs $CONTAINER | grep -B 1 initialAdminPassword; do sleep 1; done ● # Add password to instance Metadata for easier retrieval for Demo purposes gcloud compute instances add-metadata --zone us-central1-c --metadata PASSWORD=$(docker logs $CONTAINER | grep -B 1 initialAdminPassword | head -n 1) $(hostname)
- 9. What is Jenkins Pipeline : Basics Pipeline 9
- 10. Jenkinsfile Maven Terraform What is Jenkins Pipeline : Jenkinsfile Pipeline 10 Jenkinsfile Gradle Spinnaker Jenkinsfile Make Python RepositoryRepositoryRepository
- 11. Pipeline A: Maven, Terraform, Jbehave Pipeline B: Pipeline C: Gradle, Spinnaker, TestNG Make, Python, Cucumber RepositoryRepositoryRepository What is Jenkins Pipeline : Shared Libraries Pipeline 11 Jenkinsfile Pipeline A Jenkinsfile Pipeline B Jenkinsfile Pipeline C
- 12. Jenkinsfile Pipeline C Jenkinsfile Pipeline C Jenkinsfile Pipeline B Jenkinsfile Pipeline B Jenkinsfile Pipeline A Jenkinsfile Pipeline A Pipeline A: Maven, Terraform, Jbehave Pipeline B: Pipeline C: Gradle, Spinnaker, TestNG Make, Python, Cucumber RepositoryRepositoryRepository What is Jenkins Pipeline : Shared Libraries Pipeline 12 Jenkinsfile Pipeline A Jenkinsfile Pipeline B Jenkinsfile Pipeline C
- 13. Jenkinsfile Pipeline C Jenkinsfile Pipeline C Jenkinsfile Pipeline B Jenkinsfile Pipeline B Jenkinsfile Pipeline A Jenkinsfile Pipeline A Pipeline A: Maven, Terraform, Jbehave Pipeline B: Pipeline C: Gradle, Spinnaker, TestNG Make, Python, Cucumber RepositoryRepositoryRepository What is Jenkins Pipeline : Shared Libraries Pipeline 13 Jenkinsfile Pipeline A Jenkinsfile Pipeline B Jenkinsfile Pipeline C
- 14. Demo : MPL Operation 14 ● Install Library ● Add Tools ● Add Jobs ● Run ● Customize Pipeline Pipeline
- 15. Privileged and Confidential 15 MPL Structure : Features • Module overrides: project → nested lib → MPL • Module inheritance: reuse logic of overridden module • Nested library tree: to implement features for teams • Unit tests: make sure your modules will work properly
- 16. Privileged and Confidential 16 MPL Structure : Execution Process Jenkins MPL and libraries Project repository Pipeline Start Pipeline script Jenkinsfile Pipeline Custom Pipeline Custom Pipeline Custom Modules Modules DSL Code/DSL Pipeline trigger
- 17. Privileged and Confidential 17 MPL Structure : Folder Structure github.com/griddynamics/mpl /vars /src/com/griddynamics/devops/mpl /resources/com/griddynamics/devops/mpl/modules /<stage> MPLManager.groovy MPLPipeline.groovy MPLConfig.groovy Pipelines configs & poststeps controller Stage folder (“Build”, “Deploy”, ...) Module definition (“MavenBuild.groovy”, “Deploy.groovy”, “GradleTest.groovy”) MPLModule.groovy MPLPost*.groovy Interfaces <name><stage>.groovy Helper.groovy Static functions to help with @NonCPS & hard groovy /test/groovy/com/griddynamics/devops/mpl/{testing,modules} Testing base classes & modules classes contains test cases
- 18. Privileged and Confidential 18 MPL Structure : Nested Libraries MPL (fork) Nested (mlib) Company Nested (dlib) MPL (GitHub) Modules: com/company/marketing Pipelines: MPipeline → CPipeline Pipeline: CPipeline Modules: com/company/devops Modules: com/company/division Pipelines: DPipeline → CPipeline Steps: doJenkinsMagic Nested (library) Nested (library) Jenkinsfile @Library("mlib") MPipeline {} Jenkinsfile @Library("mlib") MPipeline {} Jenkinsfile @Library("dlib") DPipeline {} Jenkinsfile @Library("dlib") DPipeline {} ... ... ...
- 19. Questions 19
- 20. Thank you 20
- 21. 21 Addendum vm startup script content for the demo: # VM Details: Vanilla Centos 7 image; Open HTTP Traffic; All API permissions; # copy-paste the script below into custom startup script box in GCP; # PASSWORD metadata in instance will contain initial password for jenkins # Make sure we have docker until yum install -y docker; do sleep 5; done # Start docker service systemctl start docker # Run official blueocean jenkins image docker run -d -p 80:8080 jenkinsci/blueocean CONTAINER=$(docker ps | tail -n 1| cut -f 1 -d ' ') # Make sure Jenkins is up until docker logs $CONTAINER | grep -B 1 initialAdminPassword; do sleep 1; done # Add password to instance Metadata for easier retrieval for Demo purposes gcloud compute instances add-metadata --zone us-central1-c --metadata PASSWORD=$(docker logs $CONTAINER | grep -B 1 initialAdminPassword | head -n 1) $(hostname)
- 22. Few Words About me 22 ● MS in CS ● Decade in IT Automation ● 6 years Senior Engineer @Grid Dynamics ● Things I Like: Hiking, Fixing Cars, MTB, 3D Printing and Fabrication