Testing fundamentals in a changing world
0 likes455 views
This document discusses testing fundamentals in an agile environment. It emphasizes that testing is a team responsibility and should be integrated throughout the development process, with automated and non-functional testing. Frequent testing and integration is needed to provide early feedback and reduce dependencies. Documentation needs are reduced as testing shifts from a separate phase to being embedded in development.
Testing fundamentals in a changing world
- 1. 1 Testing fundamentals in a changing world Derk-Jan de Grood - November 2016
- 2. Aim of this session 2
- 4. CI/CD Assumptions Teams Collaborate Integration is Continue Tests are Automated Deployment is hands-off process No Automation Backlog Clear Acceptance Criteria Feedback loop to improve Testing Frequent Product Launch
- 5. 5 A test report still has value YES ! NO ! Releasing x-times a day reduces the need for formal acceptance
- 6. 6 The value of my work….
- 7. Agile Test Responsibility ‘In Agile, the accountability for the right level of quality delivered at the right time belongs to the collective team. The team should embrace the best skill sets of each contributor and plan for quality and testing at every step of the release and within each sprint.’ Michael Sowers 7
- 8. The testers mind 8 UT, ST, SIT, UAT Big Data Test techniques DocumentationHow to test Metrics IoT DefectsRisks Stakeholders Automation UX Security Test data
- 9. 9
- 10. It ain’t done until its accepted 10 Witness Development en Tes ng Tested solu on Witness report Customer
- 11. Witness report: 2 outcomes Advice to accept solution • Professional testing • Good process • Good test results Advice to not accept solution • No professional testing • No Good process • No good test results 11
- 13. In the real world Stakeholders need status info 13
- 14. 14 It ain’t done until it’s tested
- 15. Testing a Central Topic Has the train arrived at the station? 15
- 17. 17 It ain’t done until it’s tested
- 18. It ain’t done until it’s integrated 18
- 20. Architecture • What are the business processes? • What are the components? • What are the interfaces? Acceptance criteria • What is the Minimal Viable Product? • What integrations are needed to make it work? Requirements traceability • When are we complete? • How do test results add up to acceptance? 20 Missing
- 21. What should a car minimally do? 21
- 24. Ensuring Integration (rough sketch) 24 Organization Component System Service Continuously (in the sprint) Occasionally (e.g. prior to a release) Towards Value and System thinking Annotation: Increasing the system (e.g from Units tot Systems) results in less frequent integration, because it becomes harder to test the integration. This has impact on the time-to-market.
- 25. Effects 25 Witness Development en Tes ng Tested solu on Witness report Customer
- 26. 26 Reduction of depencies Early Feedback Early Error Detection Shift to Automation
- 27. How much documentation? 27 Type of Test Throw away Regression Compliance Smoke & Monitoring From: BUFD To: JIT
- 28. 29 From Silo to Team effort Methodic testing is declining
- 29. TBYDWTFI P 30 Shift to Non- functional tests Annotation: Asking the stakeholders for The Bugs You Don’t Want To Find In Production will define the test you need to do before the code goes live. No doubt stakeholders will mention non functional items like Security, UX and Performance
- 30. 32
- 31. Agile Test Strategy 33 What needs to be tested? Feedback, steering and progress Auditing the test work Coaching the testing team members Organizing tests that do not fit the sprint
- 32. Test Report 34 Lot 1 Lot 3 Lot 5 Lot 4 Lot 6 Lot 2 Tell the testing story • Do you know what is tested in the various teams? • Do you know how well they did it? • Do you have proof? • How do you rate the quality? • What concerns do you have (combine the bugs and translate to business impact)?
- 33. 37 Please, will you do some testing for me
- 34. WRAP-UP 38
- 35. Shifts Releasing x-times a day reduces the need for formal acceptance Shift to Automation and Bug hunting Shift to Business tests Towards value and systems thinking Shift to Non- functional tests From Project to Operations Methodic testing is declining From: BUFD To: JIT
- 37. 42 Integrated with development Integrated with Business Efficient and Complete Traceable Automatic Is perceived to have value
- 38. 43
- 39. Derk-Jan Valori Coltbaan 4a 3439 NG NIEUWEGEIN The Netherlands • derkjandegrood@valori.nl • +31(0)651807878 • www.valori.nl • @DerkJanDeGrood • http://djdegrood.wordpress.com 44
Editor's Notes
- #5: Teams collaborate to deliver each sprint a working product Integration is continue Tests are automated Tests are run from the build server Deployment is hands-off process TDD ensures no backlog in TA Acceptance criteria are clear Feedback loop to create better tests in place Product goes live regularly
- #6: Release advice does it have value? When releasing multiple times a day….. What is the need for a test report / advice TADAY dame claimed 400 times a day a release
- #7: Workshop where we guided a department of IM to define their roles in Agile context…
- #12: Scenario advice to not accept: nobody wants this shouldn’t be a surprise
- #13: The way to avoid the second scenario is by putting into place witness process Set of goals that need to be reached by 4 activities. Extensive reviewing Witnessing is an ongoing phase and is all about transparency and working together. Avoid surprises, mitigate risk, clear understanding on quality0 Audit: De leverancier toont aan dat: checklist van uit te voeren activiteiten, -> paper trail klopt! Bewijslast ligt bij leverancier. Demo: periodieke inzage in kwaliteit van product. Toont test werkzaamheden van afgelopen periode, testscenario’s worden afgespeeld en inzicht wordt gegeven van de belangrijkste bevindingen (wat werkt nog niet). -> Demo rapport! Meetings: Strategisch testoverleg, Finding Meeting, Demo -> Kwaliteitsoverzicht Review: Testplannen, testcases, Testresultaten, Non functional testen.
- #17: Due to the shift from projects to operations, there is no end point, it’s a continue process. Which creates a shift towards Automation and bughunting. Who needs a formal release advice/report Eet your own shit….
- #19: We do not want a screen, we want a feature
- #24: Integration becomes important if we start thinking in featues and business epics rather than system adaptions
- #25: How to ensure Integration We do: CI/CD MBT UT Automated System test Automated e2e test Interface testing Manual Regression testing Integration sprints Other General trend: Increasing the system (e.g from Units tot Systems) results in less frequent integration, because it becomes harder to test the integration. This has impact on the time-to-market and this insight might lead to targeted improvements
- #26: @Split: Technical tests could be: Exploraotry tests, Bughunting, unit test, ST testing, performance testing, interface testing, Business rule testing Business tests could be AO , e2e-chain, Testing in the large Note the Pi-shaped tester: More than one specialism, which one you chose
- #30: Dev does the testing Met a lot of developers want to the the minimal which is good, but do they oversee the total picture Testers might feel misunderstood, or alone in their quest for quality Testers matrix: keep them clicking Method testing seems to vanish Testing is not fixed in the policy but in the DOD Methodic testing is declining Does your testing still have value (in the eye of the beholder?)
- #32: The above strategy has a focus on Acceptance and getting the service, product life Should we do business testing also…. This does not happen that much What hooks do you need in the code to measure this? Is it the next stage in the agile implementation model?
- #33: When we need to take of so much items, wide spread, do we believe it will work out itself, What do we need? Yes a test strategy
- #42: Widespread: Busines involvement, Organisational readyness, Governance, product vision Integrated in the process: erverybody is testing, tests are done throughout the process and a lot of communication is involved; top keep it simple and effective Technincal : Automated, unit tests and intregrated in the build process, Fast (every day evry sprint).