Version your build process as you version your code
0 likes424 views
The document discusses the concept of literate programming as applied to build processes, emphasizing the use of natural language and code snippets to describe build steps in a version-controlled environment. It introduces the literate plugin for Jenkins, enhancements for job configuration, and practical applications in a multi-branch project context. Additionally, it contrasts the use of Markdown and YAML for environment specification, presenting the potential for improved project modeling and build management.
![Footer
#jenkinsconf
Markdown vs YAML
Environments
============
* `windows`, `java-1.6`
* `linux`, `java-1.7`
Build
=====
mvn –B clean verify
24
environments:
- [windows, java-1.6]
- [linux, java-1.7]
build: mvn –B clean verify
YAMLMarkdown
versus](https://image.slidesharecdn.com/literate-jucwest2015-150903224423-lva1-app6891/85/Version-your-build-process-as-you-version-your-code-24-320.jpg)
Version your build process as you version your code
- 1. Version your build process as you version your code Vincent Latombe @Vlatombe
- 2. Footer #jenkinsconf The speaker Software Engineer at CloudBees Heavy Jenkins user since 2010 Maintainer of the Clearcase plugin Contributes to core, git and literate 2
- 3. Footer #jenkinsconf Agenda What is Literate ? Literate in Practice Literate in real life Under the hood 3
- 5. Footer #jenkinsconf Literate Programming • Introduced by Donald Knuth in 1983 • Explain program logic using natural language with snippets of code and macros 5
- 6. Footer #jenkinsconf Literate Builds • Applies Literate paradigm to builds • Describes the build steps in a marker file or simply in README.md • Build definition stored in the same SCM as the code it is building 6
- 7. Footer #jenkinsconf Literate Builds • Marker file within the SCM • May contain the build definition • Fallback to README.md if empty 7
- 8. Footer #jenkinsconf Markdown Cheatsheet A section ========= This is text inside section. And `this is inline code`. some code 8
- 9. Footer #jenkinsconf Hello world example Hello world literate project ============================ Build ----- Let's say hello echo "Hello world" 9
- 10. Footer #jenkinsconf Ant example Environments =========== Notice how we specify the environment to build with by providing Jenkins node labels or tool installer names in code snippet sections attached to bullet points in an "environment" section? * `ant-1.8`, `java-1.7` Build ===== ant clean dist 10
- 11. Footer #jenkinsconf Matrix build? No problem Complex project =============== Environments =========== We have two different environments that the build must be run on: * `linux`, `gcc-4.2`, `ant-1.8`, `maven-3.0.5`, `java-1.7` * `windows`, `vs-pro-2012`, `ant-1.8`, `maven-3.0.5`, `java-1.7` 11
- 12. Footer #jenkinsconf Build commands per environment Build ===== We have two different sets of build instructions, one for building with visual studio and the other for building with GCC * On `gcc-4.2`, we start by building the native code ./configure make * On `vs-pro-2012`, we have a batch file to do the native steps call build-native.bat 12
- 13. Footer #jenkinsconf Additional tasks • Defined in the marker file • Enabled per branch using the job configuration • Lightweight promotion – Self-promotion – Manual promotion (with optional parameters) 13
- 14. Footer #jenkinsconf Extensions (1/2) : job configuration • Things you want as a Jenkins admin – Timestamp logs – Discard old builds – Clean up workspace before build – Notification to GitHub/Stash – Use Branch Properties 14
- 15. Footer #jenkinsconf Extensions (2/2) : in SCM • Things you leave up to the developer – Files stored in SCM under .jenkins folder – Basic: <extensionName>.xml, contains the config snippet found in config.xml – Implement Agent to expose higher-level configuration files – location of test results – coverage threshold 15
- 17. Footer #jenkinsconf Installation • Set up experimental update center • http://updates.jenkins-ci.org/experimental/update-center.json • Check for updates • Install Literate plugin and dependencies • Restart if needed 17
- 18. Footer #jenkinsconf DEMO TIME ! • https://github.com/Vlatombe/literate-sample 18
- 19. #jenkinsconf Literate in real life
- 20. Footer #jenkinsconf Context • About 3000 developers • Commercial products – ~1000 Java/JEE – ~2000 C++ • For internal tooling, add – Python – Ruby – Perl 20
- 21. Footer #jenkinsconf It’s about control! • Fully open • Locked down • Balanced – Template-based solutions: only defined attributes can be customized – Literate: commands can be customized, post-build actions can be whitelisted 21
- 22. Footer #jenkinsconf Infrastructure 22 Stash Jenkins ❑ 1.Enable CI 2.Creates 3.Index branches 4.Parses marker Literate Multi- branch project Literate branch project Project/ Repository Literate env. project Literate env. project 5.Update build status
- 23. Footer #jenkinsconf YAML Parser • Markdown considered too textual, too much space for syntax errors, even if using README.md was tempting • YAML fanboys • New parser was easy to implement thanks to existing Literate architecture 23
- 24. Footer #jenkinsconf Markdown vs YAML Environments ============ * `windows`, `java-1.6` * `linux`, `java-1.7` Build ===== mvn –B clean verify 24 environments: - [windows, java-1.6] - [linux, java-1.7] build: mvn –B clean verify YAMLMarkdown versus
- 27. Footer #jenkinsconf Literate-api • Builds a Project Model from a text file • Doesn’t depend on Jenkins • Pluggable model builders – Markdown – YAML 27
- 28. Footer #jenkinsconf Scm-api • jenkins.scm.api.SCMSource : provider for hudson.scm.SCM instances • Able to comprehend multiple heads on a SCM – Git branches – Github pull requests – Gerrit reviews 28
- 29. Footer #jenkinsconf Branch-api • Toolkit to handle multi-branch – Dead branch policy – Untrusted branches – Build retention – Throttling • Foundation for – Multi-branch freestyle project – Multi-branch template project – Multi-branch workflow (see demo by @tyvole at 1pm) 29
- 30. Footer #jenkinsconf Literate-plugin • Extension points – LiterateBranchProperty : decorate branch and environments with anything (buildwrapper, properties…) – Agent : Builds Publisher instances from files in the SCM. 30
- 31. Footer #jenkinsconf What’s next? • Integration with multi-branch workflow – Use the literate descriptor to feed a workflow – Branch sources for GitHub, Bitbucket, Stash pull requests – Additional plugins compatibility • Essentially the same problem as with workflow, without the necessary investment behind 31
- 32. Footer #jenkinsconf Takeout • Drive your build process from your SCM • Use a description markup, independant from Jenkins • Proper multi-branch support! • Give it a try ! 32
- 33. #jenkinsconf Footer Thanks to our Sponsors! 33