How Openstack is Built
0 likes7,068 views
The document details the infrastructure of OpenStack, highlighting key components such as Git, Gerrit, Zuul, Jenkins, and Nodepool, which facilitate continuous integration and testing. It provides insight into the management, configuration, and operational processes behind OpenStack's CI system, including tools like Puppet and Ansible. Additionally, it mentions the scale of OpenStack development, with over 265,000 commits and involvement from thousands of developers.
How Openstack is Built
- 1. How Openstack is Built Ant(on) Weiss - Otomato http://otomato.link 1
- 2. How Big is OpenStack? 265,850 commits 5,744 developers 54 project teams 585 git repos 2
- 3. OpenStack CI System 3 • Developed and managed by the OpenStack Infra team
- 4. Ansible+Puppet - two invisible hands. ■ All configuration of CI infra nodes is managed by Puppet manifests ■ The ‘Puppetmaster’ is actually an Ansible control machine ■ Puppetmaster holds playbook + hiera data ■ Ansible playbooks trigger ‘puppet apply’ on the nodes 4 +
- 5. Git - the mighty archivist ■ the most widely used distributed open-source VCS ■ Served by ‘cgit’ ■ Changes submitted with the help of ‘git-review’ tool 5
- 6. Gerrit – the gatekeeper ■ Originally developed by Shawn Pearce at Google for Android development. ■ Manages git repositories ■ Provides code review and approval rules ■ Collaborates with human and non- human gatekeepers 6
- 7. Zuul – the dispatcher ■ Written by James E. Blair ■ Defines and enforces the gating process ■ Listens to Gerrit events ■ Manages the queue and dependencies ■ Dispatches the verification process to Jenkins (through Gearman) 7
- 8. Zuul – the beautiful dispatcher ■ Written by James E. Blair ■ Defines and enforces the gating process ■ Listens to Gerrit events ■ Manages the queue and dependencies ■ Dispatches the verification process to Jenkins (through Gearman) 8
- 9. Gearman - the outsourcer ■ a framework to farm out work to other machines or processes 9 ■ sends jobs to Jenkins through jenkins gearman plugin (by Khai Do & James E. Blair) ■ manages the job queues
- 10. Jenkins - the butler :) ■ Continuous Integration Server ■ Executes testing jobs, records executions results ■ Jobs performed on slaves ■ “Our jenkins servers, at peak load, runs 20,000+ jobs per day. At that load we require many jenkins slaves (900+)…” (From Jenkins Gearman plugin page) ■ Jobs created and managed by Jenkins Job Builder ■ devstack slaves provisioned by Nodepool 10
- 11. Jenkins Job Builder ■ Configures Jenkins jobs using YAML files stored in git ■ Defaults, macros and job configs - job: name: example-docs node: node-label triggers: - zuul builders: - git-prep - docs publishers: - scp: site: 'scp-server' files: - target: 'dir/ectory' source: 'build/html/foo' keep-hierarchy: true - console-log 11
- 12. Nodepool - the provisioner ■ Manages a pool of Devstack images to use in project testing. ■ Provisions nodes on HP and Rackspace clouds. (using novaclient) ■ Makes sure there are always available nodes for each provider type. ■ Listens to events from Jenkins ZeroMQ publisher plugin. (by C.Boylan) ■ Tears down old instances when tests are completed. 12
- 13. DevStack Gate ■ The integration test that verifies that all projects still work together after the changes ■ Provides a full OpenStack installation on a single machine with Devstack ■ Installs Devstack and runs Tempest (the OpenStack integration test suite) 13
- 14. Logstash ■ All jenkins job logs are indexed and stored by Logstash ■ Logs pushing also handled by Gearman jobs 14 ■ Elastic Search helps analyze the test results
- 15. Wrap Up ■ Git ■ Gerrit ■ Zuul ■ Gearman ■ Jenkins Job Builder ■ Jenkins ■ Devstack ■ Nodepool ■ Logstash 15 Only 9 out of 32 projects/tools managed by the Openstack Infra Team
- 16. Some References: ■ Scaling the OpenStack Test Environment - http://amo-probos.org/post/15 ■ Zuul: a Pipelining Trunk Gating System - http://amo-probos.org/post/14 ■ Multiple Jenkins Masters with Jenkins Gearman Plugin - http:// docs.openstack.org/infra/publications/gearman-plugin ■ Understanding the OpenStack CI System - www.joinfu.com/2014/01/ understanding-the-openstack-ci-system/ ■ cgit - https://git.zx2c4.com/cgit/about/ 16