Continuous Packaging is also Mandatory for DevOps
3 likes1,031 views
Bruno Cornec, an expert in open source and Linux since 1988, emphasizes the importance of continuous packaging in the DevOps ecosystem. He introduces project-builder.org as a tool to simplify packaging for upstream projects, highlighting its benefits in consistency, reproducibility, and community engagement. The document details various packaging techniques and tools, advocating for a culture of 'package early, package always' to enhance project delivery and integration.
Continuous Packaging is also Mandatory for DevOps
- 1. Continuous Packaging is also Mandatory for DevOps Bruno Cornec HPE Open Source & Linux Strategist
- 2. Software engineering and Unices since 1988: ● Mostly Configuration Management Systems (CMS), Build systems, quality tools, on multiple commercial Unix systems ● Discovered Open Source & Linux (OSL) & made first contributions in 1993 ● Full time on OSL since 1995, first as HPE reseller then @HPE Currently: ● HPE OSL Technology Strategist, OSL Advocate and Converged Infrastructure Ambassador, WW Linux Community Lead for the HPE Open Source Profession ● POSS conference, OpenStack.fr and AFUL board member. Conferences at WW level at LinuxCon, Linux.conf.au, Fosdem, RMLL, ... ● MondoRescue, Project-Builder.org, UUWL, PUSK & python-redfish Project Lead ● LinuxCOE, mrepo, tellico, rinse, FOSSology, collectl, Ironic contributor ● FOSSBazaar/SPDX and OSL Governance enthusiast ● Mandriva, Mageia, Fedora packager And also: ● Amateur singer (Alto / Tenor), recorder player since 1976 and Choir director since 1987, CD collector (7000+), Concerts, Photography Introducing Myself
- 3. The DevOps Continuous Delivery pipeline Infrastructure as code Continuous Delivery Pipeline Version controlled Peer reviewed Autom ated tests (lots) Gates (autom ated / m anual) Autom ated deploym ent Feedback loop (m onitoring) Codifed O peration Readiness Zuul Nodepool Continuous Packaging Change in • Executable code • Confguration • Infra / environment • Data • Monitoring • … Change in • Executable code • Confguration • Infra / environment • Data • Monitoring • … Project-Builder.org
- 5. ● Startup scripts ● Specific tools ● Functional updates ● Security updates ● Community driven or Commercial (HW certification, LTS, support) Linux distribution: A project by itself ● Coherent packages set (1k-30k) taken from upstream projects ● Package Manager ● Management tools ● Installation program
- 6. RPM/deb format advantages ● Binary and source formats available w/ multiarch support ● Native support for LSB/FHS ● Provides metadata, build procedure, patches and upstream content ● Manages installation with dependencies, upgrade, removal ● Signature/Checksum support and verification ● Deployment server availability – Fully automated ● Baseline support ● RPM Package database available to query metadata ● Provides stability and coherency from the distribution Linux packages at Docker's age ? It still make sense !
- 8. “Make upstream projects life easier with regards to packaging their software” Project-Builder.org goal
- 9. “Make my life easier with regards to packaging my software” Project-Builder.org itch
- 10. ● Packaging should be a project concern as well as coding, testing, installing, .... especially for smaller projects ● Packaging as your only way of delivery (not a dream) Minimal overhead, maximum benefit: ● Consistancy and reproduceability for devs and users ● Distribution & deployment server integration, ● Consistency with distribution and avoids dependecy hell for consumers ● Packaging as a marketing activity for the upstream project. Easy way to extend your user base, and improve your community relationship and is a “competitive advantage”. ● New mantra: “Package early, package always” ● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (whatever the tool) Benefts from Continuous Packaging From Version Control System (VCS) to Packages
- 11. ● VCS agnostic: no VCS but guys it's 21st century now, SVN, CVS, Mercurial, GIT and GIT/SVN, SVK.... ● OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 150+ distro tuples made and counting – repositories for yum, urpmi, apt. Solaris pkg. ● Build environment agnostic: local, VM (QEMU, KVM...), VE (Docker, chroot, rpmbootstrap, rinse, mock, debootstrap...), RM (build farm) ● No project impact: preserves the md5sum of the delivered upstream sources. Can be completely external to the upstream project. ● Avoids duplication of code and metadata ● THE SOLUTION IS INDEED CONTINUOUS PACKAGING (with project- builder.org !) Project-Builder.org goals Support the Continuous Packaging vision by being agnostic
- 12. Continuous Packaging Architecture Project Build + metadata VM Build Local build RM Build Farm (may host VMs) Remote build Project Repository Local Build ServerPackagers Developers VE Build
- 13. Demonstration !
- 14. ● Easy creation of VE (Docker containers from distribution images) and VM. ● Macros system with perl variables to avoid duplication of metadata ● Repository management (apt, yum, urpmi) included ● Check validity of packages (lintian, rpmlint) ● Template creation for new projects, new versions management Project-Builder.org goodies Batteries included:-)
- 15. Docker images management Minimal OS Image Image Proj1 deps Image + pb tools Image Proj2 deps Proj1 container Proj2 container Phase 0 (tag: d-v-a) - newve • Reuse Image from Docker Trusted or Local Registry Phase 1 (tag: d-v-a-pb) - (sbx2|)setupve • adds pb tools and build account Phase 2 (tag: d-v-a-pb-proj) - prepve • adds project build dependencies Phase 3 (container) - build2ve • Build project in the Docker container
- 16. Web sites: ● http://www.project-builder.org ● http://trac.project-builder.org Projects using project-builder.org: ● http://www.project-builder.org (of course :-) ● http://www.mondorescue.org ● http://www.fossology.org ● http://www.linuxcoe.org Learn with: ● Examples from http://trac.project.org/browser/projects/ ● Articles from http://brunocornec.wordpress.com/tag/proj ect-builder.org Web resources Project-builder.org Related tools Project-Builder.org is mostly suited for upstream projects wanting to package their applications Distributions provide each their build tools: • SuSE: Open Build Service (Multi distro, Web based, BaaS) • Fedora (Koji) • Mageia (Youri, mgarepo...) Other complementary tools: • Buildbot • GitLab
- 17. “Changes are never easy to make. There is comfort and safety in tradition, but change must come, no matter how painful or expensive it may be.” Bill Hewlett
- 18. Thank You !Linus Torvalds, Richard Stallman, Eric Raymond, Nat Makarevitch, René Cougnenc, Eric Dumas, Rémy Card, Bdale Garbee, Bryan Gartner, Craig Lamparter, Lee Mayes, Gallig Renaud, Andree Leidenfrost, Phil Robb, Bob Gobeille, Martin Michlmayr among others, for their work and devotion to the Open Source Software cause... and my family for their patience :-) Bruno.Cornec@hpe.com Open Source and Linux Technology Strategist http://downloads.linux.hpe.com