Managing Jenkins with Python
4 likes1,056 views
This document discusses managing Jenkins jobs programmatically using Python and the Autojenkins library. It describes how to create template jobs that can be used to generate new jobs dynamically based on project metadata. New jobs are created by populating placeholders in the template with values from a project database, allowing jobs to be managed and updated automatically rather than through the Jenkins web UI.
Managing Jenkins with Python
- 1. Managing Jenkins with Python Vijay Kumar B.<vijaykumar@bravegnu.org>
- 2. Continuous Integration Every developer makes changes to the code and pushes them to the version control system. The continuous integration server pulls the code from repository runs unit tests on the code and verifies its sanity. Benefits Accidentally missed out files are detected early. Code is verified in a clean room environment. Code can be tested in various target operating systems and configurations. Build breakages and unit test breakages are immediately notified.
- 3. Continuous Integration Servers Jenkins - popular continuous integration server written in Java. Other open source continuous integration servers: BuildBot CruiseControl Drone.io GitLab CI
- 5. Jenkins Jobs A job specifies where to checkout the source from the commands to build the code what to do with built artifacts. Jobs are created through a web interface.
- 6. Fetching Source Code Specify where to get the source from. Specify whether to build periodically, or build when code is changed.
- 7. Build Steps and Post Build Actions Specify what commands are to be executed to build and run unit tests. Specify who should receive notifications, on build failure.
- 8. Job Representation No text file representation that can be edited by hand. Only a web interface to edit jobs. Job representation is dumped to XML format, using persistence APIs.
- 9. config.xml <project> ... <scm class="hudson.scm.SubversionSCM" plugin="subversion@2.1"> <locations> <hudson.scm.SubversionSCM_-ModuleLocation> <remote>https://pike.zilogic.com/svn/proj/ansible-desktop /trunk</remote> ... </hudson.scm.SubversionSCM_-ModuleLocation> </locations> ... </scm> ... <triggers> <hudson.triggers.SCMTrigger> <spec>H/15 * * * *</spec> <ignorePostCommitHooks>false</ignorePostCommitHooks> </hudson.triggers.SCMTrigger> </triggers>
- 10. config.xml (Contd.) ... <builders> ... <hudson.tasks.Shell> <command>export BUILD_VERSION=$BUILD_ID make -f .jenkins.mk before_script make -f .jenkins.mk script </command> </hudson.tasks.Shell> ... </builders> <publishers> <hudson.tasks.Mailer plugin="mailer@1.10"> <recipients>vijaykumar</recipients> ... </hudson.tasks.Mailer> </publishers> ... </project>
- 11. Problem Managing Jenkins jobs through Web UI, does not scale. Jobs are created by copying an existing job, and modifying it! Violation of the DRY rule: Don’t Repeat Yourself!
- 12. Solution Many people are trying to address the issue in various ways: Job DSL plugin - https://github.com/jenkinsci/job-dsl-plugin/ Jenkins Job Builder - http://docs.openstack.org/infra/jenkins-job-builder/ Autojenkins - https://github.com/txels/autojenkins
- 13. Autojenkins Jenkins provides REST API Create jobs Fetch config.xml of existing jobs Update config.xml Autojenkins Python package that wraps over the REST API Provides many useful funtions
- 14. Template Job A template job can be created with place holders. The job is disabled, which means Jenkins will not run it. Autojenkins can create new jobs based on the template job, and values specified for the placeholders def create_copy(self, job, template, enable=True, _force=False, **context): """ Create a job from a template job. """
- 15. Project DB Description for each project for which CI is required. name: ansible-desktop template: python-template summary: Ansible scripts for updating config. and installed packages. owner: vijaykumar branches: - 1.0 - 2.0 Used to populate the template, and create the jobs.
- 16. Per Project Build File The chunk of what is different across projects in the build steps. The build steps are put up in a Makefile called .build.mk within the project folder. The template just invokes a make target in the .build.mk. As a bonus, the CI build steps are under version control. This is similar to what Travis CI does.
- 17. Script to Update Jobs A class ProjInfo to hold the per project information. name summary owner template branches
- 18. Script to Update Jobs (Contd.) The following code creates a job for each project. jenkins = Jenkins(self._config.jenkins_url, auth=(self._username, self._password)) ci_jobname = "%s-ci" % self._proj_info.name jenkins.create_copy(ci_jobname, template_job=self._proj_info.template, enable=True, _force=True, proj_summary=self._proj_info.summary, proj_name=self._proj_info.name, owner=self._proj_info.owner)
- 19. Finishing Notes Job update script itself runs as a job under Jenkins! Manage over 150 jobs at Zilogic. Separate release and CI jobs, for each project.