Continuous Delivery (Internet-Briefing 2012-04-03)
5 likes899 views
The document discusses approaches to software deployments, noting that traditional organizations deploy software 4 times per year while newer organizations deploy software 15 times per day. It advocates automating the entire software delivery pipeline including deployments to reduce risks and costs, by applying the principle of "if it hurts, do it often" through continuous integration, delivery, and deployment.
Continuous Delivery (Internet-Briefing 2012-04-03)
- 1. Continuous Delivery Software-Deployments ohne graue Haare 3. April 2012 – Corsin Decurtins
- 2. Some numbers… 4 deployments 15 deployments per year per day bank, insurance company, social network, Web 2.0, government, transport Google Mail, Flickr, Facebook authority Netcetera | 2
- 3. Some numbers… On the left Traditional application from something like a bank, an insurance company, a government agency, a transport authority … you name it. On the right One of those new fancy social network, Web 2.0-ish application. Think Facebook, Google Mail, Flickr, … Both are serious applications with serious business value. Netcetera | 3
- 4. Some numbers… Both applications need a high up-time. Deployments to the production environment are risky. Both companies try to reduce the risk of deployments. Two completely different approaches to do that: "Banks" try to reduce the risk and effort for deployments by reducing the number of deployments. The "social networking" site tries to reduce the risk of deployments by making much, much smaller deployments … but a lot of them. Netcetera | 4
- 5. Some numbers… The questions now are: a) Which organization is right? b) Where in this range are you? c) Where in this range should you be? That's what I want to talk about today. That and why moving toward the right side might be the better choice for the future. Netcetera | 5
- 6. Deployment Installation of a software release on a target environment Replacement of existing software releases Data migration System reconfiguration Netcetera | 6
- 7. Deployment Complexity Data migration Multiple components Synchronized deployments Dependencies Netcetera | 7
- 8. Deployment Risks Server not starting Tuning issues Out of disk space Deployment Time Database migration failure Wrong dependency version Configuration typos Missing tools or libraries Communication failure Missing certificates Version mismatch Wrong access rights Bad Performance … Netcetera | 8
- 9. Cost of a Deployment Complexity Risks Staffing Time, On-Call Duty Off-Hour Work Down-Time Stability Netcetera | 9
- 10. Our goals (yours might be different) Improve Quality Fewer errors, less (unexpected) down-time, less stress. Standardization Different applications should deploy in a similar way. Makes it easier for the operations and the development teams. Faster Deployments Less down-time, smaller need for off-hour deployments. Netcetera | 10
- 11. Software Delivery Pipeline EIS / Metrics test-int test-uat prod check-in build deploy deploy deploy unit testing integration performance testing testing dev user acceptance ops security review SCM Repo CSO qa dev Netcetera | 11
- 12. Software Delivery Pipeline The Software Delivery Pipeline is the combined set of processes, procedures and tools that you use to bring code from the development environment into production. Of course you would make the pipeline even wider by including requirements engineering etc. but for the context of this talk, we will start with code. Netcetera | 12
- 13. Software Delivery Pipeline Builds Propagate through the Pipeline Successful completion of a step triggers the next steps. Errors interrupt the Pipeline In case of an error, the pipeline processing is interrupted. Expensive steps are only done if the previous steps worked For example (manual) User Testing is only done on builds that passed the automated tests. Some Pipeline steps might require manual triggering Deployment to uat or prod is probably still triggered manually Netcetera | 13
- 14. Software Delivery Pipeline A few things that you should notice right here: Everybody has a Software Delivery Pipeline. You might not have thought about it yet, it might be automated or manual, very simple or complex, but you have a pipeline. Whether you want it or not, whether you manage it or not. Your pipeline might look differently. Depends on the project, the customer, the business domain, the degree of maturity, … Netcetera | 14
- 15. Software Delivery Pipeline Coming back to our risk and cost analysis, we can identify some problem areas: Deployments are probably not tested Deployments are done manually So again, the knee-jerk reaction would be to try to reduce the number of deployments. They are expensive, risky and mean a lot of stress for the development team, the customer and the operations people. However, there are very similar issues on the left side of the diagram. We have some risky and expensive things there too (building, unit testing, deploying, integration testing, …). But the approach there is not too do it less frequently, but to automate it and do it more often. The underlying principle that we see there is … Netcetera | 15
- 16. If it hurts, do it often. Netcetera | 16
- 17. Software Delivery Pipeline EIS / Metrics test-int test-uat prod check-in build deploy deploy deploy unit testing integration performance testing testing dev user acceptance ops security review SCM Repo CSO qa dev Netcetera | 17
- 18. Continuous Integration What people did on the left side was to automate the pipeline steps and run them automatically and very frequently. Automation improves the quality by making the steps reproducible, predictable, verifiable, repeatable, … This is essentially known as Continuous Integration. A lot of companies already do this. But the automation often ends after the run of the integration tests and the creation of the build artifacts. One way of looking at Continuous Delivery is to extend the principles from Continuous Integration and bring them further down the pipeline to the right side. Netcetera | 18
- 19. Continuous Delivery Automation of Software Delivery Pipeline Automation of the Deployment Automated Triggering (Continuous Execution) Automated Testing / Verification Netcetera | 19
- 20. Continuous Delivery Automation Automation of the delivery pipeline and the deployment process to the various environments. This includes deploying code, running database migrations, configuration, restarting, … Automated Triggering (Continuous Execution) Deployments (and subsequent steps) are triggered automatically, usually by upstream pipeline steps. Automated Testing / Verification Deployments are tested and verified by automated tests. Netcetera | 20
- 21. Automation of the Deployment Process $ deploy <application> <version> <environment> Deployment was successful $ deploy <application> <version> <environment> Deployment failed, rolled back to old version Netcetera | 21
- 22. Anatomy of a Deployment Notify the running application that a deployment is happing Set up a maintenance page Stop the running application Install the new application version Run a database migration Start the new application version Reconfigure the entry server Switch off the maintenance page Netcetera | 22
- 23. Anatomy of a Deployment Consists of multiple consecutive steps Steps are reusable for different deployments Steps are reusable for different applications Might vary from version to version Might depend on the version that is currently installed Netcetera | 23
- 24. Anatomy of a Deployment Step Verify the pre-conditions Do something Validate the post-conditions Netcetera | 24
- 25. Pre-Conditions Encoding of the assumptions of a deployment step. Used to verify that the deployment step could actually work. Make sure we do not attempt a deployment step, if the assumptions are not met. Netcetera | 25
- 26. Verification Verification that a deployment step worked. Verification that a deployment step had the desired effect. Makes sure that we detect problems as early as possible in the deployment process. Netcetera | 26
- 27. Error Handling and Rollbacks Ability to roll back at any time Detecting errors Automated Rollbacks Manual Rollbacks Netcetera | 27
- 28. Error Handling and Rollbacks Ability to roll back at any time Back up the preexisting state of the system Try to make the deployment non-destructive Detecting errors Deployment process has to be able to detect errors Automated Rollbacks Very desirable But also very complex Has to be tested (continuously) Netcetera | 28
- 29. Manual Rollback It's just a workaround, but often a viable option Probability of failing deployments is relatively small We are testing the deployment continuously after all. Automated deployment process provides operations people with the necessary tools and artifacts to roll back easily Backups Non-destructive deployments Netcetera | 29
- 30. Data Migration Data migration is a tricky thing Slow Complex Error-prone Stop-the-World Automation and continuous testing helps More frequent and therefore smaller deployments help Netcetera | 30
- 31. Data Migration Patterns On-the-fly migration Read-Only availability Storage Layer abstraction NoSQL storage layer Netcetera | 31
- 32. Data Migration Patterns On-the-fly migration Application can deal with multiple versions of the database Data migration is done on-the-fly (batch job or touch-and-go) Read-Only availability Database is read-only as long as the migration is on-going Storage Layer abstraction Encapsulation of the storage layer NoSQL storage layer Less stringent requirements on schema compliance Netcetera | 32
- 33. Deployment Testing Deployments are tested continuously Continuous Testing environment Continuously, triggered automatically Pre-Production environment Regularly, triggered manually "Same" configuration as the production environment "Same" state as the production environment Netcetera | 33
- 34. Software Delivery Pipeline EIS / Metrics test-int test-uat prod check-in build deploy deploy deploy unit testing integration performance testing testing dev user acceptance ops security review SCM Repo CSO qa dev Netcetera | 34
- 35. How do we get there? It's a process Continuous delivery is not a binary thing. Continuous improvement of the maturity. Netcetera | 35
- 36. Forcing Teams Just force the teams to do continuous delivery Let them figure out how to do it Bad idea! Netcetera | 36
- 37. Better Way Provide Teams with Tools and Support Make it easy to implement continuous delivery Iterate Do not try to solve everything in the first go Iterate and implement small steps Follow the pipeline down-stream Netcetera | 37
- 38. Standardization Good way of leveraging common requirements Provide building blocks for the implementation Common deployment steps Architecture, design and implementation patterns Best practices and guidelines Make it easy for teams to follow the standards Make it possible (but not too easy) to extend the standards Netcetera | 38
- 39. Software Delivery Pipeline EIS / Metrics test-int test-uat prod check-in build deploy deploy deploy unit testing integration performance testing testing dev user acceptance ops security review SCM Repo CSO qa dev Netcetera | 39
- 40. Abstractions for Runtime Environment Common abstractions for deployment steps Allow to abstract different runtime environments OS versions and application server versions Virtualization, Private Cloud, Public Cloud Netcetera | 40
- 41. Added Values for Deployments Collect data about deployments Verify compliance with SLAs Access Control for deployments Notify stakeholders on success/failure of a deployment Configure monitoring system for deployments Netcetera | 41
- 42. Continuous Delivery and Agile Methodologies Agile Waterfall Perfect match Not really a match, but valuable anyway Automation and automated tests Continuous Testing of Deployments Continuous testing and verification Automation of very complex processes Short release and deployment cycles Delivery of hot-fixes etc. Short feedback loop Netcetera | 42
- 43. Continuous Delivery Software Delivery Pipeline Automation Reduction of Deployment Risks Continuation of Continuous Integration Quality Improvement Continuous… Faster Deployments … Builds, Integration, Testing Down-Time minimalization … Deployment, Migration Shorter time-to-production Deployment Testing Less stressful deployments Smaller and faster deployments Potential for more frequent deployments Netcetera | 43
- 44. Continuous Delivery EIS / Metrics test-int test-uat prod check-in build deploy deploy deploy unit testing integration performance testing testing dev user acceptance ops security review SCM Repo CSO qa dev Netcetera | 44
- 45. Further Reading Continuous Delivery Jez Humble, David Farley Literally the book on "Continuous Delivery". Netcetera | 45
- 46. Further "Reading" Continuous Delivery (Talk by Jez Humble, DevOps Day 2012) http://www.infoq.com/presentations/Continuous-Delivery Interview with Martin Fowler and Jez Humble on Continuous Delivery http://www.infoq.com/interviews/jez-humble-martin-fowler-cd Continuous Delivery (Blog by Jez Humble) http://continuousdelivery.com/ Netcetera | 46
- 47. Contact Corsin Decurtins corsin.decurtins@netcetera.com +41 44 247 70 70 Netcetera | 47