Accelerate Delivery: Business Case for Agile DevOps, CI/CD and Microservices
0 likes272 views
The document outlines the benefits of adopting DevOps culture and practices, emphasizing accelerated delivery and organizational performance improvements. It highlights key metrics such as increased deployment frequency, faster recovery from failures, and reduced change failure rates as indicators of high-performing IT organizations. Additionally, it discusses the evolution of microservices and continuous integration/continuous delivery (CI/CD) as critical components for achieving these outcomes.
Accelerate Delivery: Business Case for Agile DevOps, CI/CD and Microservices
- 1. ROI For DevOps Culture and Practices ACCELERATED DELIVERY A Business Case for DevOps Culture and Practices: Lean Management, Agile, DevOps, CI/CD, and Microservices BUSINESS CASE: AGILE DEVOPS, CI/CD, MICROSERVICES
- 2. – Jeffrey Immelt, CEO of GE “Every industry and company that is not bringing software to the core of their business will be disrupted.” 2
- 3. What We Are After ❖ High performing IT organizations: ❖ 200x more frequent deployments ❖ 24x faster recovery from failure ❖ 3x lower change failure rate ❖ 22% less time on unplanned work ❖ 50% less time remediating security issues 3 DevOps The Big Picture, written by Richard Seroter (Pluralsigt)
- 4. Why We Care ❖ Elite DevOps performers: ❖ 2604x faster time to recover from incidents ❖ 46x more frequent code deployments ❖ 1.53x more likely to meet of exceed business goals ❖ 1.8x more likely to recommend their team to others 4 CI/CD: The Big Picture, written by Barry Luijbregts (Pluralsigt)
- 5. [ 5 Author of best-selling agile development book Early adopter of TDD, DevOps, Agile, etc. Successfully ran development organizations Developed open source software used by millions • Java Champion 2018 Early adopter and advocate of microservices • Worked on Vert.x, QBit, Reakt, Groovy, Boon, etc. • Speaker on microservices at JavaOne • Designed/implemented microservices-based systems that scale to 100M users RICK HIGHTOWER
- 6. [ 6 ‣ Past experience ‣ Latest trends ‣ Shoulders of giants SLIDE DECK BASED ON MANY BOOKS, AND MORE
- 7. [ 7 ‣ How we got here ‣ History of • MicroServices • DevOps / Agile • CI/CD • Kubernetes ‣ Business proposition ‣ CD/CD - DevOps practices ‣ Continuous delivery • Continuous integration • Lean management and monitoring / KPIs • SCM / Version Control / GitOps / Immutable infrastructure • Trunk-based development ‣Live Demos: Concrete best practices OUTLINE
- 8. 8 DevOps is not a task; it’s a culture.
- 9. DevOps not just the name of a team ❖ DevOps is more than having a team called DevOps ❖ DevOps is a mindset and development culture ❖ Culture of DevOps, Agility, Lean, MVP, etc. is a clear win ❖ Case studies ❖ Continuous learning/Continuous improvement and growth ❖ There are guides, books, practices, and information ❖ It is a journey
- 10. Acceleration in Practice ❖ Make customers happy ❖ Deliver more ❖ Less burnout ❖ Grow the value of the company ❖ Make more money 10
- 11. Acceleration in Practice ❖ Empower developers ❖ Docker, MiniKube, Kind, K8s ❖ Create environments on the fly ❖ Eliminate manual processes ❖ Reduce people-intensive tasks ❖ Add quality to the schedule ❖ Test-driven development ❖ Avoid long lead times ❖ Avoid long-lived branches ❖ Hard to merge ❖ Hard to deploy 11
- 12. How we got here History of Microservices, Agile, and CI/CD Brief history of time
- 13. How we got here ❖ Web pages that were brochures ❖ eCommerce ❖ Legacy integration ❖ Rush to ‘webify’ businesses ❖ SOA: wrap legacy systems as services to use from the web ❖ Virtualization, Virtualization 2.0, Cloud, Containers, and now Container orchestration
- 14. Microservices: INCEPTION and Natural Evolution ❖ Now you can run a Java Virtual Machine in a Docker image ❖ Which is just a process pretending to be an OS ❖ Which is running in an OS that is running in the cloud ❖ Which is running inside of a virtual machine ❖ Which is running in Linux server that you don’t own that you share with people whom you don’t know ❖ Servers are not giant refrigerator boxes that you order from Sun and wait three months for (circa 2000)..… Goal was to run a lot of things on same server ❖ Did you develop code in the 90s with punch cards? ❖ Microservices recognize trend 14
- 15. Microservices ❖ Focus is building small, reusable, scalable services ❖ Adopt the Unix single-purpose utility approach to service development ❖ Small and malleable so they can be released more often ❖ Easier to write ❖ Easier to change ❖ Go hand in hand with continuous integration and continuous delivery ❖ Heavily REST-based and message oriented ❖ Focus on business capability ❖ Refocus on object oriented programming roots ❖ Organize code around business domains. ❖ Data and business rules colocated in the same process or set of processes. What is microservice architecture?
- 16. Microservices: Key ingredients ❖ Independently deployable, small, domain-driven services ❖ Own their data (no shared databases) ❖ Communication through a well-defined wire protocol usually JSON over HTTP (curl-able interfaces) ❖ Well defined interfaces and minimal functionality ❖ Avoiding cascading failures and synchronous calls - reactive design for failure
- 17. [ 17 CDN INVALIDATOR MICROSERVICE – INTELLIJ
- 18. [ 18 “It’s not the daily increase but daily decrease. Hack away at the unessential.” – Bruce Lee SOA EVOLVED
- 19. Microservices: Evolution of SOA ❖ SOA and Microservices have common goals and purposes ❖ Refinement to meet goals of polyglot devices and 3rd generation virtualization ❖ Cloud, container, container orchestration ❖ Parts of SOA that worked well ❖ Microservices Web technologies based (or streams or messages) ❖ To provide scalability, modular, domain-driven, small, and continuously deployable cloud-based services
- 20. [ 20 “Microservices Architecture … perhaps started out as SOA and applying lessons learned as well as pressure to: support polyglot devices, deploy more rapidly And adopt architecture liquidity that cloud computing and virtualization/ containerization provide” —Microservices Architecture Paper SOA VS. MICROSERVICES
- 21. [ 21 ‣ Philosophy behind microservices mirrors Unix ‣ Unix’s inventor, Ken Thompson, defined its philosophy: • One tool, one job. ‣ Emphasizes building short, simple, clear, modular, and extendable code • Easily maintained and repurposed by other developers MICROSERVICES: UNIX PHILOSOPHY What is microservice arc
- 22. MicroServices: Achieving Resilience ❖ Avoid synchronous calls to avoid cascading failures ❖ Instead embrace: ❖ Streams, queues, ❖ Actor systems ❖ Event loops ❖ Other async calls. ❖ Spend more time with distributed logging/log aggregation w/MDC ❖ Now distributed tracing: Jaeger, Service Mesh 22
- 23. MicroServices: Monitoring and KPIs ❖ Customer/User experience KPIs ❖ Debugging (requests per second, # threads, # connections, failed auth, expired tokens, etc.) ❖ Circuit breaker (monitor health, restarts, act/react based on KPIs) ❖ Cloud orchestration (monitor load, spin up instances) ❖ Health checks and observable KPIs 23
- 24. MicroServices: Continuous Deployment ❖ Microservices are continuously deployable services ❖ Focus of microservices is on breaking applications into small (micro), reusable services that might be useful to other services or other applications. ❖ ‘micro’ part of microservices comes to denote small ❖ Services can be deployed independently. ❖ Can be tweaked and then redeployed independently. ❖ Microservice vs monolith when deploying What is microservice a
- 25. Evolution of Agile Development DevOps, Agile, TDD, CI/CD Brief history of time
- 26. Devops: Lean, Scrum, XP, Agile, TDD, CI/CD, 12 factor ❖ TDD, CI and CI/CD ❖ Test Driven Development and Agile ❖ XP, Agile, Scrum ❖ CI/CD (Jenkins and the tools that came before) ❖ CI/CD needs automated testing ❖ DevOps aka DevSecOps ❖ Heroku and the birth of 12 factor deployment, DevOps, KPIs, SRE ❖ YBYOI vs. SRE vs. DevOps and where do you fit? ❖ SRE: Observability, Log aggregation, KPIs/Metrics, Distributed/Trace logging ❖ What is GitOps? ❖ What is immutable infrastructure? ❖ What is cloud native?
- 27. [ 27 KUBERNETES
- 28. Kubernetes (K8s) ❖ Services, stateful sets, namespaces, tags, ingress, egress ❖ Helm/Kustomize for packaging an app or set of related MicroServices and deploy to K8s ❖ Multi-cloud support and just cloud support ❖ Monitoring built-in (or at least easily pluggable) ❖ Easy to ramp up (or easier) ❖ Supports flexible deployment models ❖ Integrates with Cloud providers services or runs standalone (on prem or cloud) ❖ Container Orchestration: Yarn, Mesos, Marathon, Nomad, Borg, ECS and now Kubernetes ❖ What came before and now: Heroku/PaaS/IAAS/EC2, Docker, Docker Swarm, Mesos/ Marathon, Nomad, ECS, EKS, etc. –– K8s is the current mindshare champ. 28
- 29. No Margin. No Mission. Business case: Winning the race Why we want to do it? Why it is important!
- 30. Organizational performance ❖ High performers 2x the rate will exceed organizational performance goals as low performers: ❖ 2x profitability ❖ 2x productivity ❖ 2x market share ❖ 2x number of customers ❖ High performers twice as likely to exceed non-commercial performance goals as low performers ❖ 2x better quantity of products and services ❖ 2x operating efficiency ❖ 2x customer satisfaction ❖ 2x quality of products/services ❖ 2x achieving organizational/mission goals ❖ 50% increase in market capitalization compared to low performers! 30
- 31. Culture ❖ Bad delivery performance major contributor to burnout ❖ Deployment pain and poor software delivery practices cause organizational burnout ❖ Good technical practices predict continuous delivery ❖ DevOps Culture: Improve organizational culture, job satisfaction, delivery performance, less burnout, less deployment pain, and less time spent on rework! ❖ High performers spend 50% less time remediating security issues than low performers ❖ Trunk-based Development (like Github flow) ❖ High performers have shortest integration times and branch lifetimes ❖ Branch life and integration typically lasting hours or a day ❖ Low performers have longest integration times and branch lifetimes ❖ Branch life and integration typically lasting days or weeks 31
- 32. [ 32 DEVOPS ACCELERATING DELIVERY Source: Forsgren PhD, Nicole, Jez Humble, Gene Kim, Accelerate. IT Revolution Press, Kindle Edition.
- 33. Software delivery performance ❖ Delivery performance improves organizational performance and quality/ customer satisfaction ❖ Delivery performance is driven by investment in DevOps ❖ To improve overall software delivery performance Improve these ❖ deploy frequency, lead time, mean time to restore (MTTR), and change fail percentage ❖ Lead time improves with good version control and automated testing ❖ MTTR improves with good version control and monitoring ❖ Deploy frequency improves with continuous delivery ❖ Version control best practices needed for continuous delivery
- 34. Continuous delivery ❖ The ability to deliver ❖ Build quality in ❖ Work in small batches ❖ Automate repetitive tasks including ❖ testing & deployments ❖ Pursue continuous improvement ❖ Ownership ❖ Comprehensive configuration management ❖ Continuous integration ❖ Continuous testing You can’t skip steps. There is investment up front. Today’s speed up can be tomorrows painted yourself In a corner.
- 35. What are the steps How we get there What must we ensure to hit these amazing outcomes!
- 36. Accelerate DevOps ❖ Continuous delivery ❖ Architecture ❖ Product and process ❖ Lean Management and monitoring ❖ Cultural
- 37. Continuous delivery capabilities ❖ Implement continuous delivery / continuous deployment ❖ Version control all production artifacts ❖ Automate your deployment pipeline ❖ Implement continuous integration ❖ Use trunk-based development methods (like Github flow instead of git flow) ❖ Implement test automation ❖ Shift left on security
- 38. [ 38 Culture eats strategy for breakfast. – Peter Drucker CULTURE
- 39. Culture Capabilities ❖ Support a generative culture ❖ Encourage and support learning ❖ Encourage collaboration ❖ Make work as meaningful as possible ❖ Support and encourage transformational leadership
- 40. [ 40 What gets measured gets improved. – Peter Drucker MEASUREMENT
- 41. Lean management and Monitoring Capabilities ❖ Lightweight change approval process ❖ Monitor application and system KPIs to inform business decisions ❖ Proactively check system health ❖ Preemptively detect and mitigate problems ❖ Improve process and work within WIP limits ❖ Need visible dashboards to monitor/communicate WIP, quality, applications and systems perf ❖ Continuous improvement
- 42. Example Monitoring ❖ Hygieia ❖ Productivity dashboards ❖ Jenkins dashboards ❖ Runtime KPIs ❖ Customer KPIs ❖ Developer KPIs ❖ Grafana, Kibana, Prometheus, InfluxDB, StatsD, etc.
- 45. [ 45 PROMETHEUS – TRACKING KPIS
- 47. [ 47 HYGIEIA
- 48. Lean Management ❖ Process: Small Batches ❖ Decompose into features that allow for rapid development ❖ MVP - prototype with just enough features to proved business value or enable validated learning ❖ Quickly gather customer requirements (A/B testing, customer satisfaction surveys, etc.) ❖ Team experimentation ❖ Lean Management ❖ Change approval ❖ Proactive notification ❖ Monitoring and KPIs ❖ WIP limits, visualizing work
- 49. Delivery Vs. Deployment Continuous Delivery Software can be deployed to production at any time Continuous Deployment Software is automatically deployed to production all the time 49
- 50. Version control – SCM ❖ GitOps - keeping application code, system configuration, application configuration, and scripts for automating build and configuration in version control. ❖ Factors together predict IT performance ❖ Key component of continuous delivery ❖ Immutable infrastructure ❖ GitOps - keeping system and application config in git (versioned) correlates high with delivery performance 50
- 51. Deployment Automation ❖ Deployment automation ❖ Containers, config, immutable infrastructure ❖ Comprehensive configuration management (automation scripts), continuous integration and continuous testing ❖ Key metric of GitOps is how many diffs exists in system config from git to deploy 51
- 52. Continuous Integration Needs ❖ Continuous integration ❖ Relies on SCM and Deployment automation ❖ Good version control, GitOps, Immutable Infrastructure ❖ Relies on automated tests ❖ Unit ❖ Integration ❖ Acceptance ❖ Functional ❖ Performance ❖ Synthetic ❖ Code coverage ❖ Static analysis 52
- 53. Trunk-based development ❖ Trunk-based development ❖ Like GitHub Flow but shorter lived branches ❖ Fewer active branches that never outlive a sprint ❖ Branch-off master per feature, bug fix, etc. ❖ More PRs more often ❖ No code freezes; integration periods less than a day ❖ Polar opposite of git flow ❖ Easy to see WIP 53
- 54. Shift left on security ❖ Integrating security into design and testing phases ❖ Security reviews of applications, including the infosec team in design and demo process ❖ Using pre-approved security libraries and packages, and testing security features as part of automated testing suite 54
- 55. Concrete steps Knowing the road Best practices applied
- 56. Prefer Microservices… ❖ Monoliths can speed up MVP and prototypes but at a cost ❖ Hockey Stick ❖ J-Curve ❖ Monoliths make make CI/CD slower ❖ Monoliths make automated tests suites harder to build and they are needed for CI/CD ❖ Smaller monoliths and SOA is a move in the right direction, but monoliths should be considered technical debt ❖ Refactoring to microservices is a journey ❖ Adoption is a Journey ❖ Fits the CI/CD well ❖ Fits small batch well (Lean Management)
- 57. Embrace Observability from the start ❖ Log aggregation ❖ Time series data base ❖ Log all KPIs for clusters ❖ Log all KPIs for applications and services ❖ Alerting ❖ Know when and how to employ distributed tracing ❖ Distributed/Trace logging ❖ Service Mesh
- 58. [ 58 Cloud Native DevOps with Kubernetes John Arundel and Justin Domingus “The goal of an observability team is not to collect logs, metrics or traces. It is to build a culture of engineering based on facts and feedback, and then spread that culture within the broader organization.” – Brian Knox (Digital Gear) RECOMMENDED READING View on Web
- 59. What is a PR? And how to ensure quality with it ❖ A PR is a pull request ❖ PR gives other developers a chance to review code before it committed to master ❖ PR via small batch (why small? JIRA story or even a task or two) ❖ With GitHub and WebHooks you can block PRs from merging ❖ Code coverage met, build works, unit tests run, other checks via Jenkins ❖ Reviewed and approved by at least two people 59
- 60. [ 60 ‣ Unit tests ‣ Perf testing JMH ‣ Functional tests • At the HTTP layer • At the Spring Boot layer ‣ Acceptance tests ‣ Smoke integration tests ‣ Full integration tests ‣ Synthetic testing ‣ Code Coverage (sonarqube) ‣ Security/Vulnerability dependency license check • Aqua, Fortify ‣ Full integration perf testing Aqua, Fortify ‣ Full integration perf testing TESTS TO CREATE: TDD
- 61. [ 61 ‣ CI/CD ‣ Deploy often (daily or more) ‣ Test often (after every checkin run all automated tests) ‣ Block PRs from merging until they pass tests ‣ Block PRs until they are reviewed ‣ Block PRs until they reach a certain code coverage ‣ You can't do CI/CD without automated testing ‣ Testing allows you to move quickly with confidence CI/CD TO ENFORCE QUALITY
- 62. Embrace small batch work ❖ Goal of three PRs per week ❖ Goal of one to two tasks from Jira per PR ❖ Use Jira # in commits and PR comments ❖ Break stories and features up into tasks ❖ Check in interim stories ❖ Use feature flags if it is hard to break up a feature or story
- 63. Using Docker, Helm, Kubernetes ❖ Uber dev tools ❖ Persistent cluster set up ❖ Helm install Kubernetes ❖ Easy to integrate locally ❖ Docker, docker-deploy, helm, etc. ❖ Local integration possible and repeatable ❖ Same Docker containers in integration, development and prod 63
- 64. Example Automated deployment ❖ Merging into master triggers a deploy to integration and sends a Team message ❖ Approval from Product Manager pushes code to Prod or Demo ❖ Checking into main branch from PR (Trunk based development, i.e., GitHub flow) ❖ Artifacts and scripts for deployment checked into git and should not be modified (GitOps) ❖ Puts code staging area to be checked by Product Manager into containers and deployed to cluster (some ephemeral some not) ❖ Once checked by Infosec and Product Manager: ❖ Canary Deploys to 3 to 5% of traffic and is monitored (end goal) (Service Mesh, Itsio) ❖ Then more and more as it is monitored and is ok
- 65. [ 65 DEVELOPMENT PROCESS Developer Feature Work CI/CD Job Groom a Jira Feature Create Feature Branch from Master Work on Feature Commit and Push WIP Create Feature PR Respond to Peer Reviews Respond to Failed Tests Sync from Master Merge to Master Build and Test Deploy
- 66. [ 66 CI/CD PROCESS CI/CD Deployment Developer Feature Work Tag Release Deploy to Integration Run Integration Tests Jenkins Demo Workflow Jenkins Approval Workflow Deploy to Demo Environment Product Manager Ensure Changes Approve Changes w/ Jenkins Workflow CI/CD Build Run Build and Unit Tests Ensure Build and Unit Tests Ensure Code Coverage Ensure Semantic Version Ensure Release Notes Deploy to Ephemeral Integration Env. Run Integration Tests Ensure Security Checks Ensure Two Peer Reviews Ready to Merge Merge to Master Create Feature PR
- 67. Demo ❖ Jenkins ❖ Minikube/Kind ❖ Spring Boot ❖ Konfigure ❖ Kubernetes
- 68. Conclusion
- 69. Main Takeaways ❖ You can’t afford not to transform ❖ Transformation requires a deep understanding of practices ❖ ROI is there ❖ Faster to market, less burnout, more scalable, higher market cap, faster recovery, etc. ❖ Having a team called DevOps is not doing DevOps per se ❖ Continuous learning/Continuous improvement and growth ❖ Everyone gets into the DevOps mindset around TDD and integration tests ❖ Deploy small iterations continuously- Start by striving for daily deployments ❖ Automated deployment pipelines ❖ TDD and behavior driven development a MUST ❖ Kubernetes, Docker, Feature flags, Trunk based development and Microservices support these efforts ❖ DevOps is not a task; it is a culture
- 71. [ 71 1. Microservices by Martin Fowler and James Lewis 2. Microservices Architecture by Chris Richardson 3. Micro services – Java, the Unix Way by James Lewis 4. Microservices, or How I Learned To Stop Making Monoliths and Love Conway’s Law by Cliff Moon 5. Micro service architecure by Fred George 6. Microservices are not a free lunch by Benjamin Wootton 7. Antifragility and Microservices by Russ Miles 8. The Unix Philosophy 9. Conway’s Law 10. Amazon Architecture 11. Migrating to microservices by Adrian Cockroft 12. Microservices with Spring Boot 13. Microservices for the Grumpy Neckbeard 14. Microservices definition on Wikipedia 15. Microservices and DevOps by Adrian Cockcroft 16. Building and Deploying Microservices - Bart Blommaerts 17. Microservices on the JVM - Alexander Heusingfeld 18, Microservices Shaun Abrams 19. Microservices Rick Hightower 20. Microservices Monitoring Rick Hightower 21. Microservices vs. SOA Rick Hightower 22. Microservices and Docker Rick Hightower REFERENCES
- 72. [ 72 ‣ Not a new thing ‣ Not necessarily cool or hip ‣ Obvious next step in evolution ‣ Web, cloud, mobile (polyglot devices), server virtualization ‣ OS containerization, container orchestration, multi-core servers ‣ Cheaper and cheaper RAM ‣ 64 bit computing ‣ 10GB Ethernet, 100GB Ethernet, etc. MICROSERVICES ADDRESS INDUSTRY EVOLUTION
- 73. [ 73 QUALITY - UNPLANNED WORK Source, Forsgren PhD, Nicole. Jez Humble, Gene Kim, Accelerate . IT Revolution Press. Kindle Edition.
- 74. [ 74 ‣ Gather and implement customer feedback ‣ Make the flow of work through the system visible ‣ Work in small batches ‣ Foster and enable team experimentation PRODUCT AND PROCESS CAPABILITIES
- 75. [ 75 ‣ Use loosely coupled architecture • Release new services on demand without outages ‣ Empower the team to select tools; trust team to pick the best tools ARCHITECTURAL CAPABILITIES TO ACCELERATE
- 76. [ 76 ‣ Loosely coupled, well-encapsulated architecture drives IT performance. • 2017 dataset biggest contributor to continuous delivery was loosely coupled, well- encapsulated architecture ARCHITECTURE
- 77. [ 77 ‣ Experimental approach to product development highly correlates with continuous delivery ‣ Lean product development capabilities predict improvements in organizational culture like reduced burnout higher software delivery performance and overall organizational performance LEAN PRODUCT MANAGEMENT CAPABILITIES
- 78. [ 78 1. Microservices by Martin Fowler and James Lewis 2. Microservices Architecture by Chris Richardson 3. Micro services – Java, the Unix Way by James Lewis 4. Microservices, or How I Learned To Stop Making Monoliths and Love Conway’s Law by Cliff Moon 5. Micro service architecure by Fred George 6. Microservices are not a free lunch by Benjamin Wootton 7. Antifragility and Microservices by Russ Miles 8. The Unix Philosophy 9. Conway’s Law 10. Amazon Architecture 11. Migrating to microservices by Adrian Cockroft 12. Microservices with Spring Boot 13. Microservices for the Grumpy Neckbeard 14. Microservices definition on Wikipedia 15. Microservices and DevOps by Adrian Cockcroft 16. Building and Deploying Microservices - Bart Blommaerts 17. Microservices on the JVM - Alexander Heusingfeld 18. Microservices Shaun Abrams ADDITIONAL READING