JOIN 2022: Patching 3rd party software Like a boss
- 1. 04/10/2022 PATCHING 3RD PARTY SOFTWARE LIKE A BOSS Using Renovate, Tekton and ArgoCD
- 2. 2 WHO AM I? o Consultant at Ordina o Software architect & developer background o Interested in: Automation, public cloud in enterprises, hyperscale software o Company blog & LinkedIn
- 3. 1. Why? 2. What? 3. How? 4. Demo Time! AGENDA
- 4. 4 USING THE LATEST FEATURES Why?
- 5. 5 PREVENTATIVE MAINTENANCE Upgrade from 6.8.21 to 8.0.1 or from 8.0.0 to 8.0.1
- 7. o Knowing when there is an update o Knowing what to update o Knowing how to update o Validate that the update works o Rollout to production 7 WHAT IS NEEDED
- 8. 8 HOW DOES IT WORK
- 10. 10 THE PROCESS
- 11. 11 HOW DOES IT WORK Java project repo/ ├─ src/ ├─ Dockerfile ├─ pom.xml
- 12. 12 HOW DOES IT WORK Java project Docker: amazoncorretto:18.0.0 Maven: org.apache.logging.log4j:log4j-core:2.12.2
- 13. 13 HOW DOES IT WORK Java project Docker: amazoncorretto:18.0.0 Maven: org.apache.logging.log4j:log4j-core:2.12.2 amazoncorretto:18.0.1 org.apache.logging.log4j:log4j-core:2.18.0
- 14. 14 HOW DOES IT WORK Java project Docker: amazoncorretto:18.0.0 -> 18.0.1 Maven: org.apache.logging.log4j:log4j-core:2.12.2 -> 2.18.0
- 15. 15 HOW DOES IT WORK Java project Docker: amazoncorretto:18.0.0 -> 18.0.1 Maven: org.apache.logging.log4j:log4j-core:2.12.2 -> 2.18.0 PR: amazoncorretto: 18.0.1 PR: log4j: 2.18.0
- 16. 16 HOW DOES IT WORK Java project PR: amazoncorretto: 18.0.1 PR: log4j: 2.18.0 Build: amazoncorretto: 18.0.1 Build: log4j: 2.18.0
- 17. 17 HOW DOES IT WORK Java project PR: log4j: 2.18.0
- 18. 18 HOW DOES IT WORK DONE!? Coffee time!
- 19. 19 HOW DOES IT WORK DONE!? Coffee time!
- 20. 20 HOW DOES IT WORK
- 21. 21 HOW DOES IT WORK
- 22. 22 HOW DOES RENOVATE WORK Deployment deploy-repo/ ├─ resources/ │ ├─ deployment.yaml ├─ configs/ │ ├─ service.properties ├─ kustomization.yaml
- 23. 23 HOW DOES RENOVATE WORK Deployment apiVersion: apps/v1 kind: Deployment metadata: name: my-awesome-service spec: selector: matchLabels: app: my-awesome-service template: metadata: labels: app: my-awesome-service spec: containers: - name: my-awesome-service image: my-awesome-service:0.0.0
- 24. 24 HOW DOES RENOVATE WORK Deployment apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - resources/deployment.yaml - https://github.com/pietervincken/my-other-service?ref=0.0.0
- 25. 25 HOW DOES RENOVATE WORK Deployment Kubernetes: image: pietervincken/my-java-service:0.0.0 Kustomize: resource: github.com/pietervincken/my-other-service:0.0.0
- 26. 26 HOW DOES RENOVATE WORK Deployment Kubernetes: image: pietervincken/my-java-service:0.0.0 Kustomize: resource: github.com/pietervincken/my-other-service:0.0.0
- 27. HOW DOES RENOVATE WORK Deployment Kubernetes: image: pietervincken/my-java-service:0.0.0 -> 0.0.1 Kustomize: resource: github.com/pietervincken/my-other-service:0.0.0 -> 0.1.0 27
- 28. HOW DOES RENOVATE WORK Deployment my-java-service:0.0.0 -> 0.0.1 my-other-service:0.0.0 -> 0.1.0 PR: my-java-service: 0.0.1 PR: my-other-service: 0.1.0 28
- 29. HOW DOES RENOVATE WORK Deployment PR: my-java-service: 0.0.1 PR: my-other-service: 0.1.0 29
- 30. 30 HOW DOES IT WORK DONE!? Coffee time!
- 31. 31 HOW DOES IT WORK DONE!? Coffee time!
- 32. 32 HOW DOES IT WORK
- 33. 33 HOW DOES IT WORK
- 34. HOW DOES IT WORK Deployment Kubernetes: image: pietervincken/my-java-service:0.0.1 Kustomize: resource: github.com/pietervincken/my-other-service:0.1.0 34
- 35. HOW DOES IT WORK Deployment Kubernetes: image: pietervincken/my-java-service:0.0.1 Kustomize: resource: github.com/pietervincken/my-other-service:0.1.0 Repository changed 35
- 36. HOW DOES IT WORK Deployment Build kustomization 36
- 37. HOW DOES IT WORK Deployment Apply changes 37
- 38. 38 HOW DOES IT WORK
- 39. 39 HOW DOES IT WORK DONE! Succes!
- 40. 40 DEMO TIME
- 41. o Inform about the update o Build confidence in update sources o Docker hub o Maven central o Internal sources o Start with low impact o Limit amount of concurrent updates 41 CONVINCE THE BRASS Baby steps
- 42. 42 CUSTOMIZE IT It’s all just config
- 43. • Conventional commits • At least semver based tags • Good automated testing • -> high level of confidence for auto-merge • Clear deployment process to (pre-) production • Automation and configuration as code are key 43 NOT REQUIRED, BUT GOOD TO HAVE
- 44. • Renovate Github • ArgoCD documentation • Tekton documentation • Tekline: Pipeline as code solution for tekton • Demo project • Ordina JWorks 44 RESOURCES
- 45. 45
- 46. PIETER VINCKEN COMPETENCE LEAD CLOUD T +32 (0)15 29 58 58 E pieter.vincken@ordina.be
Editor's Notes
- #5: https://www.dutchnews.nl/news/2022/05/mark-rutte-deleted-text-messages-from-his-old-nokia-daily-vk/ https://thefintechtimes.com/onepoll-how-outdated-technology-affects-working-from-home-productivity/ https://techtalk.currys.co.uk/computing/workplace-productivity/
- #6: Example of Log4Shell / Log4J vulnerabilty TODO Embracing updates and upgrading frequently allow for small changes And very little effort to update. Big bang upgrades are costly.
- #7: TODO Upgrading is inevitable. Having an (automated) process in place is key https://www.comparitech.com/blog/information-security/cybersecurity-vulnerability-statistics/ https://owasp.org/Top10/A06_2021-Vulnerable_and_Outdated_Components/ Too this day, still unpatched log4shell systems available…
- #8: Renovate -> not only what to update but also how -> Changelog!
- #12: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #13: Fictive example!
- #14: Fictive example!
- #15: Fictive example!
- #16: Fictive example!
- #17: Fictive example!
- #18: You can allow Renovate to merge automatically on successful builds, but in general and definitely when starting to use this, you’ll merge these PRs by hand.
- #19: Grab a coffee and scroll through some Reddit posts about how automation makes your life easy!
- #20: Fictive example!
- #23: Dependency detection: https://docs.renovatebot.com/modules/manager/ Docker-compose, Ansible, Kubernetes, Flux, Helm, kustomize, Terraform, Tekton(?!?) Supported repository systems https://docs.renovatebot.com/#supported-platforms Github, Bitbucket, Gitlab, … Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #26: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #27: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #28: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #29: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #30: Either automated or manually Moves through your normal development process E.g. deploy it to a development environment first. After that’s validated, merge to main for production Prevent copy paste errors by allowing renovate to update the different environment repositories
- #31: Fictive example!
- #32: Fictive example!
- #35: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #36: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #37: Dependency detection: https://docs.renovatebot.com/modules/manager/ Supported repository systems https://docs.renovatebot.com/#supported-platforms Renovate: Scans your repositories to detect package files and their dependencies Checks if any newer versions exist Raises Pull Requests for available updates
- #38: yaml file extension by Grafix Point from https://thenounproject.com/browse/icons/term/yaml-file-extension ArgoCD performs API calls based on the YAML it has generated.
- #40: The only 2 manual actions we had to do was merge 2 PRs. Everything else was automated.
- #42: Outdated versions Security vulnerabilties Only update when new feature needed Update all depedencies at once -> not a good idea Small updates, validated, one by one -> way to go
- #43: Possibility to limit type of updates Major / minor / patch Limit what to update In/exclude update sources (E.g. only internal container registry) Auto-merge trusted sources E.g. in-house maintained libraries Auto-merge low impact targets E.g. supporting application Limit number of PRs Grouping similar updates Limiting PRs by design Custom/additional upgrade steps through scripts
- #46: Zalando’s, Netflixes and Tesla’s of the world are build to change. The fast moving companies are able to adapt quickly and efficiently The key to move fast is to often in small steps. Big leaps rarely work out.