CLUG 2014-10 - Cookbook CI with Jenkins
Download as PPTX, PDF2 likes2,224 views
This document discusses using Jenkins as a continuous integration server for Chef cookbooks. It covers installing and configuring Jenkins and plugins to run Foodcritic, Chefspec, Test Kitchen, and publish cookbooks and artifacts. Pipelines are created using the Build Pipeline and Job DSL plugins to test and promote cookbook releases through different environments. While configuration sometimes requires XML, Jenkins is mature, extensible, and can immediately provide value by catching problems early and keeping infrastructure automated and consistent.
CLUG 2014-10 - Cookbook CI with Jenkins
- 1. Cookbook CI with Jenkins Part 1
- 2. Who Am I? • "zts" most places on the internet • Mostly Ops, Background, Mostly • Co-founder Elastera
- 3. Who Are You?
- 4. Cookbook Testing • foodcritic, chefspec, test-kitchen • Get things right • Keep things right
- 6. CI Servers • System for running actions on every commit • Identify problems early • Jenkins, GoCD, TravisCI, Wercker, etc
- 7. Why Jenkins? • Mature • Extensible • Good Chef Cookbook • Free
- 8. Why Not Jenkins? • TMTOWTDI • XML, so much XML • Automation is not a joy
- 9. Jenkins Cookbook • Install Jenkins & Plugins • Create jobs • Run commands and scripts
- 10. Previously… • Cooking With Jenkins • ...and Kitchen, and Docker
- 11. Subsequently… • Federated Jenkins with Chef • github:erichelgeson/jenkins-chef-dsl/
- 12. Basic Configuration • Install Plugins • Configure Plugins • Configure Jenkins
- 13. Installing Plugins • Idempotent • No dependency resolution • Restart may be required
- 14. Plugin Configuration • XML :( • ...containing module version strings :(( • Hold nose and use cookbook_file
- 16. Creating Jobs • jenkins_job • Job DSL plugin • Jenkins Job Builder
- 19. Pipeline • Foodcritic • Chefspec • Kitchen • Publish
- 20. And after publish? • That's why this is Part 1...
- 21. Foodcritic • Warnings plugin :) • XML config :(
- 24. Chefspec • yarjuf gem • Publish JUnit test result report
- 27. Test Kitchen • kitchen-docker • Matrix jobs • EnvInject and AnsiColor plugins
- 29. Berkshelf • upload to Chef server • record version as artifact
- 31. Berkshelf • curl -s https://jenkins.example/job/berks-upload/ lastSuccessfulBuild/artifact/VERSION
- 32. Gluing it together • Publish Artifact • Copy Artifact • Build Pipeline
- 33. Pipelines • Build Pipeline plugin • Build Graph View plugin • Build Flow DSL • ...and more
- 34. Build Pipeline
- 35. What's Missing? • JUnit output from serverspec • View most recent run of all pipelines • Notifications
- 36. Not the whole story... • Testing a set of cookbooks • Testing multiple nodes • Promoting releases
- 37. Summary • Not that hard to get started • Immediately valuable • Share your work!
- 38. Questions?
- 39. Resources http://www.cryptocracy.com/blog/2014/01/03/cooking-with-jenkins-test-kitchen-and-docker/ https://erichelgeson.github.io/blog/2014/05/10/automating-your-automation-federated-jenkins-with-chef/ http://atomic-penguin.github.io/blog/2014/05/16/stupid-jenkins-tricks-part-2-cookbook-pipelines/ http://www.hurryupandwait.io/blog/using-git-to-version-stamp-chef-artifacts https://github.com/jenkinsci/job-dsl-plugin/wiki https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin https://wiki.jenkins-ci.org/display/JENKINS/Building+a+matrix+project
- 40. Thankyou Twitter: @zts email: zts@elastera.com
Editor's Notes
- #3: Yes, we’re hiring.
- #4: Let’s get the “hands up” out of the way… Who tests their cookbooks, chefspec and/or kitchen? Who practices TDD for their cookbooks? Who automatically runs tests on commits?
- #5: Not here to sell you on TDD, but it’s worth trying if you haven’t. Can encourage better design, as well as building a library of useful tests.
- #6: Addresses a problem on large, shared codebases - parallel streams of activity, on branches, taking a lot of effort to merge. Not such a huge problem on single cookbook repo’s (in my experience). However, the tooling to support this is useful.
- #7: With a good set of tests, prevents a whole class of errors caused by haste/impatience/divergent dev envs/etc.
- #9: Also worth mentioning - pipelines are something of an afterthought
- #11: Early this year, I demonstrated how to use Chef to configure Jenkins to run cookbook tests. This demonstrated some of the things I’m using today, but it was pretty rough. It was already due for an update two weeks after it was published (thanks to a new Jenkins cookbook), and I haven’t gone back to it yet…
- #12: Meanwhile, Eric Helgeson wrote a great post (with example cookbook) showing how he was using Jenkins. This includes groovy scripts for configuring authentication, and use of the Jenkins DSL plugin (which we’ll come to later). This motivated me to have another crack at the problem.
- #13: Add slides with examples of each
- #15: This needs to be a better way to solve this, but it wasn’t immediately apparent and pragmatism triumphed.
- #16: Idempotent? Well, you’ll get the same result but it will run every time…
- #19: Screenshot
- #20: If you’ve used these tools, you’ll know that foodcritic is very fast, chefspec is relatively fast, and kitchen tests are relatively slow. If your chefspec tests fail, you won’t even bother running Kitchen. Only if everything passes will you publish that commit to the Chef server.
- #24: Screenshot
- #28: Why not use kitchen parallel runner? Documented as intended for interactive use. Hard to read interleaved output.
- #33: What’s an artifact? For compiled languages, a build step compiles the source code, the resulting artifact is passed to subsequent jobs. In my pipeline, the primary artifact is Berksfile.lock
- #34: Others? Build Graph(?), Build Flow DSL (?), etc
- #37: Subjects for a later talk…
- #38: Sharing useful nuggets makes it easier for others to get started and contribute. Special thanks (again) to Eric Hegelson for this.
- #41: Add contact details