Boost your testing power with Exploration
2 likes1,995 views
The document discusses exploratory testing. It defines exploratory testing as an approach that emphasizes personal freedom and responsibility of testers to continually optimize their work by treating learning, test design, and execution as parallel activities. The document provides strategies and techniques for exploratory testing, including using test charters, coverage outlines, risk lists, and test logs. It also discusses how more exploration can boost testing value by focusing on what needs to be done, creating engagement, leveraging tacit knowledge, and using insights to inform subsequent tests. Mastering exploratory testing requires practice, pairing with others, debriefing sessions, and training to generate test ideas quickly.
Boost your testing power with Exploration
- 1. Exploration Boost your testing power with Exploratory Testing!
- 2. • What is exploratory testing? • Strategy and techniques of ET • SBTM: plan, structure & traceability • How more exploration can boost the value of your testing • How to master exploratory testing?
- 3. Exploratory Testing “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project”
- 4. Exploratory Testing “An approach to software testing that emphasizes the personal freedom and responsibility of each tester to continually optimize the value of his work by treating learning, test design and test execution as mutually supportive activities that run in parallel throughout the project." "Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next."
- 5. Exploratory Testing “An approach to software testing that emphasizes the personal freedom and responsibility of each tester to continually optimize the value of his work by treating learning, test design and test execution as mutually supportive activities that run in parallel throughout the project." "Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next."
- 6. freestyle fragmentary test pure scripted exploratory case (scenarios) vague scripts charters roles Freedom Pre- scription Source: Telling Your Exploratory Story by Jon Bach
- 7. Fairy tales Do not apply ET if: • Higher demands on demonstrability / reporting (eg. imposed standards) • Critical functionality (failure = big damage) • Inexperienced testers • Test cases must be able to be executed by a different tester • Test cases have to be re-usable • No direct feedback from test execution (the results are not immediately available) • Tests require a lot of preparation • Testing needs to be on the critical path as short as possible Source: TMap Next, chapter 14
- 8. Fairy tales Source: Fan booklet Exploratory testing (Microsoft)
- 9. Fairy tales
- 10. • What is exploratory testing? • Strategy and techniques of ET • SBTM: plan, structure & traceability • How more exploration can boost the value of your testing • How to master exploratory testing?
- 11. Exploration: searching for value and risk execution discovery design investigation reporting learning Source original picture: Two futures of testing by Michael Bolton
- 12. Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission
- 13. Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission
- 14. Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission
- 15. Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission
- 18. Source: Rapid Software Testing by James Bach & Michael Bolton This slide is used with kind permission
- 19. Stories Product Story: How the product works and how it fails... Testing Story: What we have tested and how, what we do not test and how good our testing was...
- 20. • What is exploratory testing? • Strategy and techniques of ET • SBTM: plan, structure & traceability • How more exploration can boost the value of your testing • How to master exploratory testing?
- 21. Heuristics Test Ideas Risk Catalog Issues Bugs Session General Sheets Daily Test Sessions Test Log Test Wrap-up / Planning Ongoing Session Debrief (weekly) Test planning Charters Test Log Test Wrap-up / session Session Debrief Test Log Test Wrap-up / Session Debrief Test Plan Coverage Risks Low Tech Dashboard Strategy Project
- 23. Source: Ruud Cox, Testing in a Medical Device Context, EuroStar 2012
- 24. Heuristics Test Ideas Risk Catalog Issues Bugs Session General Sheets Daily Test Sessions Test Log Test Wrap-up / Planning Ongoing Session Debrief (weekly) Test planning Charters Test Log Test Wrap-up / session Session Debrief Test Log Test Wrap-up / Session Debrief Test Plan Coverage Risks Low Tech Dashboard Strategy Project
- 25. Test Charters Explore (target) Explore Application X import With (resources) menu. Identify important features with the goal of To discover (information) developing a coverage outline and a risk list. My mission is to test Read Chapter 4 of the product <insert risk here> to specification. Prepare a mind map, and <insert coverage here> discuss it with Peter (programmer) and David (architect). The charter is a one- to three-sentence mission for a testing session
- 26. Heuristics Test Ideas Risk Catalog Issues Bugs Session General Sheets Daily Test Sessions Test Log Test Wrap-up / Planning Ongoing Session Debrief (weekly) Test planning Charters Test Log Test Wrap-up / session Session Debrief Test Log Test Wrap-up / Session Debrief Test Plan Coverage Risks Low Tech Dashboard Strategy Project
- 27. Test log: my notes Preparation: talk to product owner Test Strategy & notes on a bugs Issue/question & coverage outline Debrief using PROOF
- 28. Session sheet • Charter • Coverage areas (product areas, product elements, quality criteria or test techniques) • Start Time + Session duration (long, normal, or short) • Tester Name(s) • Charter/opportunity time , expressed as a time breakdown: – Test design and execution (as a percentage of the total on-charter time) – Bug investigation and reporting (as a percentage of the total on-charter time) – Session setup (as a percentage of the total on-charter time) • Data Files • Test Log (notes the tester made during testing) • Bugs (a problem that threatens the value of the product) • Issues (a problem that threatens the value of the testing process) Source: An Exploratory Tester’s Notebook by Michael Bolton
- 29. • What is exploratory testing? • Strategy and techniques of ET • SBTM: plan, structure & traceability • How more exploration can boost the value of your testing • How to master exploratory testing?
- 30. More exploration can boost the value of testing • Do what really needs to be done • Create engagement: help people use their brains • Take advantage of tacit knowledge and skill • Use insights from experiments to inform the next • Beware of the creative power of exploration
- 31. • What is exploratory testing? • Strategy and techniques of ET • SBTM: plan, structure & traceability • How more exploration can boost the value of your testing • How to master exploratory testing?
- 32. Challenges? Exploratory Testing = not easy and needs (a lot) training and practice Often seen as most difficult: – Note taking – Test ideas – Coverage reporting – Managing ET
- 33. How to master exploratory testing? • Just do it! • Practice, practice and practice some more… • Pair with colleagues • Testing Dojo • Debrief & retrospect • Train creating test ideas fast • Learn more about observation, experiments, biases, social science, ....
- 34. Source: Gitte Ottosen – Many flavours of exploratory testing (Agile Testing days 2012)
- 37. “A tester knows that things can be “Testing is about different” questioning & learning under conditions of Jerry Weinberg fundamental uncertainty.” “I've participated and organized RST many testing competitions. I've never seen someone win such a competition by writing down tests. Doesn't that tell us If you cannot trust your testers, you do not make them something?” write more detailed test case. But you train them! James Bach Rikard Edgren – EuroStar 2012 Gitte Ottosen – ATD 2012 Image: http://corkboard.me
- 39. huib.schoots@codecentric.nl +31 6 - 24641033 @huibschoots www.magnifiant.com
Editor's Notes
- #2: http://www.developsense.com/blog/2010/01/exploratory-testing-is-accountable/How More Exploration Can Boost the Value of Your Testing”“How Becoming More Conscious of Your Exploration Can Boost the Value of Your Testing
- #7: Scripted testing is not the "normal" way of testing. People think it is only because they are lying to themselves and lying to us. Or else they are just ignorant about testing. Anyone who is truly and only following a script is NOT TESTING. They are FAKING IT.
- #8: TMAP zegt:In welk geval kun je Exploratory Testing (beter) niet toepassen,Wanneer er hogere eisen aan de aantoonbaarheid/verslaglegging van testen worden gesteld, bijvoorbeeld door opgelegde standaards.Bij kritische functionaliteit waarvan het falen grote schade kan veroorzaken.Wanneer testgevallen moeten kunnen worden uitgevoerd door een andere tester of een testtool.Voor onervaren testers, omdat deze niet zonder ondersteuning van een expliciete techniek testgevallen kunnen opstellen.Wanneer er geen rechtstreekse feedback van de testuitvoering is, zodat testresultaten niet direct beschikbaar zijn.Bij tests die veel voorbereiding vergenWanneer het testen zich zo kort mogelijk op het kritieke pad van het project moet bevinden.http://www.softwaretesten.net/testmethodieken/tmap-next-te/samenvatting-tmap-next-hoofdstuk-14/
- #9: TMAP zegt:In welk geval kun je Exploratory Testing (beter) niet toepassen,Wanneer er hogere eisen aan de aantoonbaarheid/verslaglegging van testen worden gesteld, bijvoorbeeld door opgelegde standaards.Bij kritische functionaliteit waarvan het falen grote schade kan veroorzaken.Wanneer testgevallen moeten kunnen worden uitgevoerd door een andere tester of een testtool.Voor onervaren testers, omdat deze niet zonder ondersteuning van een expliciete techniek testgevallen kunnen opstellen.Wanneer er geen rechtstreekse feedback van de testuitvoering is, zodat testresultaten niet direct beschikbaar zijn.Bij tests die veel voorbereiding vergenWanneer het testen zich zo kort mogelijk op het kritieke pad van het project moet bevinden.http://www.softwaretesten.net/testmethodieken/tmap-next-te/samenvatting-tmap-next-hoofdstuk-14/
- #29: An Exploratory Tester’s Notebook
- #31: is that SKILLED and VISIBLE exploratory testing boosts the power of the UNSKILLED and SECRETIVE exploratory testing that everyone is already doing
- #36: Trish Khooexploratory testing as a set of learnable techniques?It’s like learning the guitar. You can learn the chords if you want, but that’s not how you become a good guitarist. At some point you have to start figuring out how to make the guitar work for you, how to get it to do what you want. Trish & piano: lessons and practice the same songs over and over again, but never enjoyed playing the pianountil I started experimenting with my own methods of learning. Music is not a mindless, step-following activity. Nor is programming. Nor is testing.