DevOps Fest 2020. Kohsuke Kawaguchi. GitOps, Jenkins X & the Future of CI/CD
- 1. GitOps, Jenkins X & Future of CI/CD Kohsuke Kawaguchi | Co-CEO, Launchable, Inc. kkawaguchi@launchableinc.com | @kohsukekawa
- 3. “software delivery is an exercise in continuous improvement, and our research shows that year over year the best keep getting better, and those who fail to improve fall further and further behind.” - Nicole Forsgren
- 4. Building and scaling high performance technology organisations
- 5. • Common platform in all clouds • AWS, Azure, GCP • OpenShift, CloudFoundry • Functionalities • Cluster scheduler • Service discovery • Load balancer • Extensibility New “Cloud Operating System” is here
- 7. DevOps is the new legacy
- 9. GitOps
- 11. Why? • Transparency without sacrificing auditability & access control • How/who to get changes in • What has happened • Tools agnostic – works with any ”Infrastructure as Code,” not just K8s
- 12. Why? • Promote higher abstraction and reuse • Review/test before you merge • Trivial roll back, change detection, etc through familiar workflows of Git/GitHub
- 13. You can have fun tinkering…
- 14. Or you can just get productive
- 15. Jenkins X vision • Figure out the best practice of how to CD cloud native apps • Not just build, test, but reviewing, promotion, changelog, collaboration, etc. • Integrate best of the bleed software in this ecosystem to achieve it • Democratize it by building a pleasant CLI that represents high-level steps • Be opinionated on how to do things • Kubernetes is a means to the end
- 17. Development Flow We Preach
- 18. More Best practice we preach • Use cloud to develop, keep your laptop for what it needs to do • Keep master always releasable • Deploy often and in small increments • Inform other people about where changes are
- 19. Best Practices That Are “Boring”
- 20. Integrate best of the bleed software in this ecosystem to achieve it
- 21. MultiCloud Support: jx create cluster
- 22. • brew tap jenkins-x/jx • brew install jx • jx create cluster Let’s get going
- 23. • You got all that tools, preconfigured properly • Jenkins, with elastic build agents to build containers • Artifact repository to speed up your builds • Monocular to catalog Helm charts • You got staging & production environments • For all your future apps to be onboarded to Jenkins X What just happened?
- 24. • Create a new project • jx create spring • Make your existing app work in Jenkins X • jx import Prepare your app
- 25. • Source code • Wiring for build, deploy, & CD • Jenkinsfile, Dockerfile, helm chart, … • All services configured for you • Repo on GitHub, webhook to Jenkins, Jenkins jobs • Initial release & all running in staging What just happened?
- 26. • Work locally on a change • Create a PR on GitHub • Have the change reviewed & merged Let’s work on a change
- 27. • Your PR gets automatically built & tested • App deployed to a PR specific ‘preview environment’ • Allows stakeholders to interact with the app • Click a link in PR to see it • (Then you merge your change) • New master gets automatically built & tested • New release gets created with changelog What just happened?
- 28. • jx promote --version v0.0.3 --env production • Go to GitHub and merge ‘deployment’ PR • (v0.0.3 appears in production) Let’s promote a release to production
- 29. Future of CI/CD
- 30. • Defend stability through depth, not just tests GitOps is necessary, but not sufficient for higher velocity
- 31. • Feature Flags • Circuit Breaker • Canary Deployment • Better Observability How do you defend in depth?
- 32. Leverage Production Traffic • Because there’s nothing like production • Traffic mirroring • A/B testing
- 34. • Situation • You are the DevOps team of a BigCo • Massive modularized codebase with web of dependencies • Big, time consuming tests around them • Questions • I want to cut cost & time of the software delivery process Smarter testing
- 35. Step 1: Dependency Analysis
- 36. • ML model predicts useful subset to run • Based on information about changes • Of 105 changes/mo, 1% is used to train the model • Impact • Only a third of tests are selected • Misses just 0.1% of broken changes • AWS cost is cut by half 36 Step 2: Predictive Test Selection
- 37. • Situation • You are the SRE team in a BigCo • You oversee 100s of apps • ~1 deployment/app/day • Questions • Can we flag risky deployments beforehand? 37 Deployment Risk Prediction
- 38. • Train model • With 40,000 deployments of which 100 are failures • Attributes: app names, commit messages, … • Impact • Predict 99% of failures • 5% false alarm rate 38 What they have done
- 39. • Learning • Most outages are estimated as “low risk” by developers • Most outages had short time span till approval • Long-maintained code is more risky • Imagine what you can do with this! • Require somebody be on call • Restrict window of deployment 39 What they have done
- 40. Where This Takes Us
- 41. • Every organization is trying to get better at software delivery • Kubernetes is an enabling technology but it can be a detractor • What’s important is DX, which is GitOps • Jenkins X brings that for K8s without you rolling your own which becomes legacy • CI/CD is continuously evolving, so is Jenkins Wrap Up
- 42. Thank You!
Editor's Notes
- #3: Every organization is trying to get better at software delivery
- #5: Not either or. Both.
- #7: So everyone is rebuilding their new devops flow
- #8: But wait, that is the new legacy Building it is hard enough, who is going to maintain it?
- #9: Better question to ask is, what is the right user experience
- #14: You need to learn the technologies Install & operationalize Kubernetes Survey the ecosystem and pick up additional tools & services Move to containers Figure out how to deploy containers to Kubernetes Wire up automation … while figuring out the practice Figure out the delivery practice What’s this GitOps thing? How do I deal with secrets? Am I doing it right? What am I missing out? (Image CC0 by https://pixabay.com/en/car-engine-prius-c-motor-vehicle-231213/)
- #17: Jenkins is like a vast road network / Jenkins X is like a high speed rail (image: https://flic.kr/p/smze69 CC-BY-SA 2.0)
- #21: Prow, monocular, Helm,
- #24: None of this is rocket science, but just think about how much it takes if you wire this together yourself (image: https://pixabay.com/en/home-building-residence-1353389/ CC-0)
- #26: (image: https://pixabay.com/en/log-cabin-cottage-house-home-1886620/ CC-0)
- #27: You can even develop locally and have the change deployed to your own development environment instantaneously
- #28: (image: https://pixabay.com/en/country-house-building-home-540796/ CC-0)
- #33: (Image: Pixabay license: https://pixabay.com/photos/cyclist-person-people-men-bicycle-3202481/)
- #34: Once you incorporate learning into your way of work, where ML can apply becomes a lot more obvious And companies are already doing this (Image: no attribution required: https://pixabay.com/illustrations/artificial-neural-network-ann-3501528/)
- #36: Diamond: source file / Square: module / Circle: test
- #37: File types, frequency of changes in the files modified, # of files modified, historical failure rate of tests, test module name, distance from changes to tests
- #41: https://pixabay.com/photos/fantasy-feet-knight-landscape-2546204/