Pipeline as Code
2 likes1,030 views
Kris Buytaert advocates for defining infrastructure and pipelines as code using tools like Jenkins Job DSL and Git. This allows infrastructure to be version controlled and centrally managed, with reusable jobs that can be updated in sync. Defining pipelines as code stops operators from manually "clicking" in interfaces and ensures consistency across teams.
Pipeline as Code
- 1. Pipeline as CodePipeline as Code Kris Buytaert @krisbuytaert
- 2. Kris BuytaertKris Buytaert ● I used to be a Dev,I used to be a Dev, ● Then Became an OpThen Became an Op ● CTO and Open Source ConsultantCTO and Open Source Consultant @@inuits.euinuits.eu ● Everything is a freaking DNSEverything is a freaking DNS ProblemProblem ● Evangelizing devopsEvangelizing devops ● Organiser of #devopsdays,Organiser of #devopsdays, #cfgmgmtcamp, #loadays, ….#cfgmgmtcamp, #loadays, ….
- 3. cAmscAms ● Build AutomationBuild Automation ● Test AutomationTest Automation ● Infrastructure as CodeInfrastructure as Code ● Monitoring as CodeMonitoring as Code ● Security as CodeSecurity as Code ● Pipeline as CodePipeline as Code
- 4. Jenkins as ExampleJenkins as Example • Open Source toolOpen Source tool • Widely adopted in the EnterpriseWidely adopted in the Enterprise • Plenty of plugins availablePlenty of plugins available • UI FocusedUI Focused
- 5. PipelinesPipelines ● Lots of themLots of them ● Similar ones, but not identical onesSimilar ones, but not identical ones ● One project = different deploymentOne project = different deployment targetstargets ● People move teams expect samePeople move teams expect same patternspatterns ● Mostly unmaintanedMostly unmaintaned
- 8. Scaling PipelinesScaling Pipelines ● Create a Pipeline,Create a Pipeline, ● For job in PipelineFor job in Pipeline • Create new Job Based on OldJobCreate new Job Based on OldJob ● Update One JobUpdate One Job ● Never refactor the restNever refactor the rest
- 9. Generating Jenkins PipelinesGenerating Jenkins Pipelines ● Template the XMLTemplate the XML ● Put it in PuppetPut it in Puppet • Worked for stable pipelinesWorked for stable pipelines • KindaKinda ● XML gets rewritten on the flyXML gets rewritten on the fly
- 10. JenkinsJobBuilderJenkinsJobBuilder ● First Usable AttemptFirst Usable Attempt ● Python BasedPython Based ● Openstack CommunityOpenstack Community ● Limited FunctionalityLimited Functionality ● Little Adoption / OpenstackLittle Adoption / Openstack Abandonned itAbandonned it
- 11. PipelineDSLPipelineDSL ● JenkinsfileJenkinsfile ● Inside a repoInside a repo ● GroovyGroovy ● Limited functionalityLimited functionality ● Initially buggyInitially buggy ● Popular for Easy tasksPopular for Easy tasks • (straight, no splits etc)(straight, no splits etc)
- 12. Jenkins Job DSLJenkins Job DSL ● GroovyGroovy ● FlexibleFlexible ● Well SupportedWell Supported ● Suitable for more complex PipelinesSuitable for more complex Pipelines
- 13. SeedjobsSeedjobs ● GroovyGroovy ● GitGit ● Rebuild jobs onRebuild jobs on commitcommit ● Projects inProjects in foldersfolders
- 14. ExampleExample
- 15. Example dsl projectExample dsl project job("${foldername}/dashing-dashboard-build")job("${foldername}/dashing-dashboard-build") job("${foldername}/dashing-dashboard-test")job("${foldername}/dashing-dashboard-test") job("${foldername}/dashing-upload2repo")job("${foldername}/dashing-upload2repo") job("${foldername}/dashing-dashboard-job("${foldername}/dashing-dashboard- deploy")deploy") job("${foldername}/dashing-dashboard-job("${foldername}/dashing-dashboard- deploy-prd")deploy-prd") job("${foldername}/dashing-build")job("${foldername}/dashing-build")
- 16. Larger CI StacksLarger CI Stacks ● Generate Pipelines / Jobs based onGenerate Pipelines / Jobs based on config files ,config files , ● Build librariesBuild libraries • CheckoutJobCheckoutJob • DeployJobDeployJob • PackageJobPackageJob ● Use Groovy / JobDSL to generateUse Groovy / JobDSL to generate PipelineDSLPipelineDSL
- 17. Other Tools :Other Tools :
- 18. Problems solvedProblems solved ● One job per task, no reuse of jobs withOne job per task, no reuse of jobs with different parametersdifferent parameters ● All Jobs are in sync, update one =All Jobs are in sync, update one = update allupdate all ● Centrally managed jobs (git)Centrally managed jobs (git)
- 19. Stop Clicking , Write CodeStop Clicking , Write Code
- 20. ContactContact Kris BuytaertKris Buytaert Kris.Buytaert@inuits.beKris.Buytaert@inuits.be Further ReadingFurther Reading @krisbuytaert@krisbuytaert http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/ http://www.inuits.be/http://www.inuits.be/ InuitsInuits EssensteenwegEssensteenweg 3131 BrasschaatBrasschaat BelgiumBelgium 891.514.231891.514.231