Packaging a Python application after you messed up - Roman Prykhodchenko
Download as PPTX, PDF0 likes48 views
The document discusses the challenges of packaging a Python application, particularly in the context of maintaining support for both community and in-house versions, often resulting in broken builds and troubleshooting delays. It proposes a solution involving the creation of a single delivery pipeline that packages shared artifacts while considering both community and private extensions. The paper highlights the use of tools like dh_virtualenv for building deb packages and emphasizes the importance of gradual migration and configuration management in Docker images.
Packaging a Python application after you messed up - Roman Prykhodchenko
- 1. Packaging a Python application after you messed up Roman Prykhodchenko, Allegro @romcheg me@romcheg.me November 2019, Warsaw www.devopsdays.pl
- 2. DevOpsDays Warsaw 2019 • Maintained and used in house • Heavily modified django-admin • Python + HTML + JS • Open-source + private extensions • In house: Ubuntu + Docker image • In community: mostly Ubuntu + deb package but.... DCIM system: Ralph 2
- 3. DevOpsDays Warsaw 2019 Problem • Community builds are often broken • Troubleshooting takes more time than we have • No motivation to support the community 3
- 4. DevOpsDays Warsaw 2019 Reasons • Two separate delivery pipelines • TravisCI: Debian package built and published once a week • Bamboo: Docker image with private extensions available on demand • Different artefacts in community and local packages • Supporting the community is not the team's #1 priority 4
- 5. DevOpsDays Warsaw 2019 Solution: Improve the delivery pipeline by re-thinking packaging. 5
- 6. DevOpsDays Warsaw 2019 • ... a file with an .rpm or .deb in the end • ... a python wheel • ... a tarball with a source code and a Makefile • ... a container image?! 0_o Package is... 6
- 7. DevOpsDays Warsaw 2019 Package is a distributable set of idempotent alternations bringing desired artefacts and state changes to a target system. 7
- 8. DevOpsDays Warsaw 2019 Containerisation is a form of packaging – the rules apply. 8
- 9. DevOpsDays Warsaw 2019 Delivery pipeline Source code 0 1 0 1 0 1 1 1 0 Build toolchain Artefacts Build dependencies Runtime dependencies Packaging toolchain Package 9
- 10. DevOpsDays Warsaw 2019 Target platforms MutableImmutable • Artefacts • Configuration • Dependencies • Artefacts • Configuration • List of dependencies Runtime depencencies Fetch 10
- 11. DevOpsDays Warsaw 2019 Target platforms Runtime dependencies are available in vendor repositories. Runtime dependencies are shipped within the package. 11
- 12. DevOpsDays Warsaw 2019 • Supporting the community was not #1 priority 12
- 13. DevOpsDays Warsaw 2019 Solution: 1. Create a single pipeline for the community and ourselves 2. Pack the same artefacts into different packages 13
- 14. DevOpsDays Warsaw 2019 Target platforms 14
- 15. DevOpsDays Warsaw 2019 0 1 1 0 0 1 1 0 15
- 16. DevOpsDays Warsaw 2019 • cPython 3.4+ • Database drivers + other shared libraries • Required python libraries • ... • Django-* Runtime dependencies 16
- 17. DevOpsDays Warsaw 2019 Deb package Artefacts: 1. Python code 2. Static files 3. Configuration Runtime dependencies: 1. python-3 2. mysql-common 3. ... 4. python3-mysql 5. python3-django 6. python3-* 17
- 18. DevOpsDays Warsaw 2019 Never run pip install on a target system. 18
- 19. DevOpsDays Warsaw 2019 • Copy dependencies to the source tree • Keeping wheels in the source tree • Package a virtual environment 0_o • No source-code changes required • Use standard management tools • Gradual migration to system packages Other options? 19
- 20. DevOpsDays Warsaw 2019 • Create one • Install python stuff • Required packages • Source code • Fix the symlinks • Pack the virtualenv into the deb package Virtualenv 20
- 21. DevOpsDays Warsaw 2019 dh_virtualenv does all of that for you! debian/control: 21
- 22. DevOpsDays Warsaw 2019 dh_virtualenv does all of that for you! debian/rules: 22
- 23. DevOpsDays Warsaw 2019 • Use all matching requirements from the vendor's repository • Use dh_virtualenv • Put the rest of the requirements along with the source code into the virtualenv • Gradually migrate the code to the libraries available in the vendor's repositories Deb package – summary 23
- 24. DevOpsDays Warsaw 2019 Docker image – extra layer of complexity • Supporting different container orchestrators • Configuration without breaking immutability • Performing operations without entering running containers • Serving static files 24
- 25. DevOpsDays Warsaw 2019 • Single entry point script • Avoid exposing the filesystem structure • Acts like a facade for all entry points • By default starts the service Startup and operations 25
- 26. DevOpsDays Warsaw 2019 • Select few essential configuration options and define environment variables for each • The entry point script puts the values of those variables into configuration files • Those in need of supplying advanced configuration should mount the entire configuration as a volume Configuration 26
- 27. DevOpsDays Warsaw 2019 • Build a separate image with static files • Use a lightweight image like nginx as a base • Users will route requests to one of the containers depending on the container orchestrator Static files 27
- 28. DevOpsDays Warsaw 2019 • Custom entry point acting like a facade • Essential configuration options as environment variables • Advanced configuration is done by mounting configuration as a volume • Static files are available in a separate image • Publish both images at the same time Docker image – summary 28
- 30. DevOpsDays Warsaw 2019 Support opinionated scenarios based on in-house usage. 30
- 31. DevOpsDays Warsaw 2019 Improve things gradually. 31
- 32. DevOpsDays Warsaw 2019 Be ready to say goodbye to corner-case users to make the vast majority happy. 32
- 33. DevOpsDays Warsaw 2019 Does not matter what decisions you make, unless they are irreversible. 33
- 34. DevOpsDays Warsaw 2019 Thank you! 34
- 35. Roman Prykhodchenko, Allegro @romcheg me@romcheg.me I'm waiting for your feedback! You can rate speakers and lectures using our official conference app
Editor's Notes
- #2: Let me just brief you quickly with the background -- I am an engineer in a team supporting certain part of the technical platform at our company and our team had a problem!
- #3: Running our services requires an extensive infrastructure built with a few thousands of assets. In order to handle that we have created a DCIM Ralph with the main goal to be the software that fulfils our internal needs. At some point of time it was published under apache license just because... somehow it gained some popularity
- #5: For the company Ralph is not a product therefore the team's focus must stay on the in-house needs, including maintenance of other systems. Open-sourcing this sort of software is debatable, yet here we are and there's no way back.
- #8: distributable --
- #9: That means all the constraints apply.
- #11: Immutable -- you should supply runtime dependencies Mutable -- you may fetch dependencies from a trusted source
- #13: Then we gave it another thought and realised the cornerstone of the mess is the last issue and as the matter of fact it's the only one from the list we cannot fix since it is a direct consequence of the business model of the company. Following the DevOps philosophy means taking responsibility for more then just the development, so we could not ignore that.
- #14: The basic idea was that whatever is given to the community should be also used by us.
- #15: That required to make several unpopular, yet necessary changes in the initial plan -- and the first of them was to sacrifice the idea of RPM builds. Moreover -- we had to select only one Debian-based distribution of GNU/Linux which is used in the vast majority of use cases.
- #17: Django application often ends up being dependent on a ton of libraries from the Django ecosystem and the combination often works only when all requirements have very specific range of versions.
- #18: External python libraries are available in canonical's repositories under certain names. Canonical maintains those packages and provides patches and security updates Yet those libraries have specific versions that won't match the requirements of the python code. There's a way to install python libraries -- pip install/easy_install -- installs from PyPi/other repos. With django that's a huge problem.
- #19: Python package management is non deterministic -- requirements do not have strict versions, you never know what gets installed on the target system. Even if you freeze the dependency tree: Installing python packages may require build tools to compile extensions written in C. Random failures We have to bring those requirements inside the package.
- #20: huge source tree, long installation, more c-libraries required, changes in the source code may be necessary to fix imports Complicated installation process, more steps to update requirements, cannot use standard python package management tools Seems funny, but... Good as a transitional stage
- #21: Therefore it is necessary to add some sort of script to the pipeline that will do the following steps: Sounds like too much of a hustle for a transitional stage.
- #25: When designing a container image, it is necessary to keep in mind that "docker run" is not how containers are often run on production. Container orchestrators like "Kubernetes" or even simple "docker-compose" require some work to be done before an image can be actually used. Docker isolates what's inside, yet it's often necessary to get under the hood, e.g. to configure the software or to perform different operations.
- #28: Since static files are inside the container, serving them won't be as easy
- #31: To summarise – when open-sourcing your software while the human-power is limited or when priorities do not put community to the 1st place follow these 3 strategies: Well done opinionated software is much better than weak one that provides many options.
- #32: Trying to start with a "by the book" approach is likely to be too time consuming or too expensive.
- #33: Be ready to receive tons of negative feedback or anger.