Continuous delivery test strategies
Download as PPTX, PDF1 like499 views
The document outlines strategies for continuous delivery (CD) and automated testing to enhance software quality and reduce delivery times. Key points include the importance of delivering code to end-users rapidly, the need for automated testing to catch defects early, and various testing strategies such as unit testing and performance testing. The author emphasizes collaboration through versioned tests and the necessity for thorough documentation in the testing process.
Continuous delivery test strategies
- 1. CD test strategies 1 26 October 2017, Hylke Stapersma, @hylke1982
- 2. 2 AGENDA CD test strategies Why continuous delivery Why (automatic) testing CI/CD Testing strategy Functional testing strategies Non functional testing strategies
- 3. Hylke Stapersma Software craftsman, Java, CI/CD, DevOps. Streaming data 3 Personal introduction
- 5. Code is value Every line of code should be focused on delivering value as soon as possible. This means the code should be delivered to the end-user within the smallest time window possible. 5 Why continuous delivery
- 6. Small increments Take small steps that can be released more often, instead of big steps that have a high risk of failing. 6 Why continuous delivery
- 7. Higher quality & quality build in Automation makes process more reliable, instead of having a human making mistakes automation needs to be setup once right instead. 7 Why continuous delivery
- 8. Faster time to market Software only makes money in production, this can only be done with a strategy that delivers software automatically as fast as possible. Reliable delivery is key here. 8 Why continuous delivery
- 9. Faster feedback Almost know instantly if the software has defects and get notified on these defects. 9 Why continuous delivery
- 10. Chain of trust Every step/test will add (automatically) add trust to a version of a application. 10 Why continuous delivery
- 12. End-user should not experience defects End-users who experience defects in a production environment will loose confidence in the product. Trust is key in making or saving money with software. 12 Why (automatic) testing
- 13. Find defects early How sooner a defect is found how cheaper it will be to fix it. A defect in production can have severe cost and cause reputation damages. 13 Why (automatic) testing
- 14. Documentation Tests descriptions and results are great ways to document how the system supposed to work. Automation can give the ability to create this completely automatically. 14 Why (automatic) testing
- 15. Ready for refactoring Having a automated test setup gives confidence to refactor and find regression problems when changing the software. 15 Why (automatic) testing
- 16. Performance Being able to say something about how the system is going to performance under certain loads instead of finding it out by end-users. 16 Why (automatic) testing
- 17. Security Find security problems in early stage. Security problems can lead up to penalties and reputation damages. 17 Why (automatic) testing
- 19. Automate everything Automation is a great way to eliminate unnecessary manual repetitions. Run all tests with a minimal set of commands. 19 CI/CD testing strategy
- 20. Code and version everything Writing tests as code offers a great way to collaborate as a team, because code can be versioned, reviewed and changed by multiple versions. 20 CI/CD testing strategy
- 21. Adopt TDD strategy Write tests before writing code. This helps to get a better understanding of the problem, and code is covered by test before being written. 21 CI/CD testing strategy
- 22. Align Choose a test framework that aligns with the solution being build. For example choose the same programming language for application and tests. 22 CI/CD testing strategy
- 23. Automate documentation Every step that runs test should create documentation automatically, use, share and distribute this documentation in a automated fashion. 23 CI/CD testing strategy
- 24. Fail fast – fail early Validate if every condition is met before starting ‘expensive’ tests. For example see if application is available or there is enough disk space. 24 CI/CD testing strategy
- 26. Test pyramid Use the testing pyramid for a fast but complete test setup. Try to cover almost everything with ‘cheap’ unit tests and only do ’expensive’ the necessary functional/acceptance tests. 26 Functional testing strategy
- 27. Unit testing One unit tests should focus a single atomic part of the application in a certain state. (good practice is one assert per test). Execute thousands per second and use a TDD approach. 27 Functional testing strategy
- 28. Mutation testing To validate the validity of unit test use mutation testing. See if tests fail when a line of code is changed. Will prove the strength of the unit tests. 28 Functional testing strategy
- 29. Property based testing Instead of example use properties to validate. For example everything in a range is true. This will validate the range. 29 Functional testing strategy
- 30. Integration/component testing Validate if software components work together. 30 Functional testing strategy
- 31. Contract testing Validate if interfaces are compliant on the server and on the client side. Share documentation about assumptions of a API. Use PACT for example. 31 Functional testing strategy
- 32. Functional / Acceptance testing Describe on a high level assumptions how to end-user will interact with the system. Use a language that can be understood by machine and humans. 32 Functional testing strategy
- 33. Manual testing Only focus manual testing on things that are hard to understand for a computer. For example does look right and does the response feels right. 33 Functional testing strategy
- 35. Security testing Based on known security issues try to break the application. 35 Non functional testing strategy
- 36. Performance testing Find the limits, normal behavior, stress behavior and sizing for the application. Works great by adopting a cloud strategy. 36 Non functional testing strategy
- 37. Compliance testing Automatically validate if the system is compliant with internal and external rules. 37 Non functional testing strategy
- 38. Chaos monkey / Resilience testing By introducing stress or failure find weak spots within a system. 38 Non functional testing strategy
- 39. 39 Thank you
Editor's Notes
- #7: Build up software and gather feedback
- #35: Baseline testing Compliance testing Documentation testing Endurance testing Load testing Localization testing and Internationalization testing Performance testing Recovery testing Resilience testing Security testing Scalability testing Stress testing Usability testing Volume testing