Continuous Delivery & Testing Madrid AfterTest
Download as PPTX, PDF1 like1,166 views
This document discusses continuous delivery and the testing challenges it presents. It describes a large financial institution's goal of delivering updates to their digital payment platform weekly. Initially they had a 9 month release cycle with many sequential test phases. The author proposed reducing features per release and incorporating more testing into development. This faced resistance from test managers concerned about quality and independent testing. Through education and demonstrations, the approach was modified to have functional testing after development and the culture shifted to prioritize automated testing and quality as everyone's responsibility.
Continuous Delivery & Testing Madrid AfterTest
- 2. @petemar5hallhttp://leansoftwareservices.com Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time. Its about building Quality in!
- 3. @petemar5hallhttp://leansoftwareservices.com Do I work in a continuous delivery model? • Your software is deployable throughout its lifecycle • Your team prioritises keeping the software deployable over working on new features • Anybody can get fast, automated feedback on the production readiness of their systems any time somebody makes a change to them • You can safely release fully tested software at anytime
- 4. @petemar5hallhttp://leansoftwareservices.com Why should you care about this delivery model? Apart from the obvious… More and more companies are asking for this experience More and more companies are trying to achieve some aspect of continuous delivery
- 5. @petemar5hallhttp://leansoftwareservices.com What test challenges does CD give us? • Speed of delivery - How can I test everything? • Emphasis on automation over use of manual procedures - who does all this automation? • How do I understand and get good test coverage? • How do you do regression testing?
- 6. @petemar5hallhttp://leansoftwareservices.com In context • My experience of shaping testing practices on a large financial enterprise project trying to achieve a continuous delivery model
- 7. @petemar5hallhttp://leansoftwareservices.com The scene A large financial institution New product - Digital Payment Platform
- 8. @petemar5hallhttp://leansoftwareservices.com Their Goal Deliver updates to a digital payment platform every week - allowing them to compete by adding and changing features on demand - allowing them to safely and quickly react to issues
- 9. @petemar5hallhttp://leansoftwareservices.com Initial State • 9 Month release cycle • 200+ development team (20+ teams) • Poor agile implementation • Lots of issues (communication, politics, etc) • Numerous sequential test phases (dev, qa, functional, performance, integration, business acceptance, operational, pre production)
- 10. @petemar5hallhttp://leansoftwareservices.com Dev Production Business Acceptance Partner Release Test Operational Test Functional Test Platform Integration Test QA Performance Test • Very traditional waterfall model • Some agile practices Initial Test & Release Model 9 Months
- 11. @petemar5hallhttp://leansoftwareservices.com Some of the initial problems • Most tester were from a more traditional testing background (scripted micro managed testing) • Very long feedback loops between development and test • Each test team in each phase had to re analyse requirements • Configuration on each test environment was different • Test management team opposed to change
- 12. @petemar5hallhttp://leansoftwareservices.com Proposed model Dev Production1 Week Business Acceptance Most test phases & practices to happen in “development”
- 13. @petemar5hallhttp://leansoftwareservices.com Approach • Reduce the amount of features going through the release process • Build as much testing into the build process as possible • Focus testers away from documentation and creating test cases, apply a context driven approach • Incorporate as many of the post development test phases as possible
- 14. @petemar5hallhttp://leansoftwareservices.com Approach (Cont.) • Use crowd testing (bug bashes) with the whole team • Invest heavily in configuration management and environment provisioning • Bring business owners close to the teams • Training on agile thinking and lean engineering practices
- 16. @petemar5hallhttp://leansoftwareservices.com “Quality is the responsibility of everybody.”
- 20. @petemar5hallhttp://leansoftwareservices.com • How is it possible to do all that testing.. • what will my 20 test teams do? • the quality will be terrible! • Where is your independent testing?
- 21. @petemar5hallhttp://leansoftwareservices.com How is it possible to do all that testing.. • Education • Demonstrations • Invite the authors of books & well known testers to do workshops
- 22. @petemar5hallhttp://leansoftwareservices.com What will my 20 teams do? • Reduce the number of teams, you don’t need that many testers • Move your remaining testers into the development teams • Explain the release test team anti pattern
- 23. @petemar5hallhttp://leansoftwareservices.com The quality will be terrible! • I say prove it… • Over 18 months, critical defects reduced from >100 per release when we started to < 2 per release at the end
- 24. @petemar5hallhttp://leansoftwareservices.com Where is your independent testing? • What does that actually mean? • What is the value of an independent test team?
- 25. @petemar5hallhttp://leansoftwareservices.com Functional/Platfor m Integration test The compromise Dev Production4 Weeks Business AcceptanceMost test phases & practices to happen in development
- 26. @petemar5hallhttp://leansoftwareservices.com To conclude • We changed the culture • We changed the perception of what testing means • Remove scripted testing, and decreased test team size • Placed responsibility of quality on everybody • We operated as a team
- 27. @petemar5hallhttp://leansoftwareservices.com Useful resources • https://leanpub.com/buildqualityin • http://codedetective.blogspot.com • http://continuousdelivery.com