Exploratory testing part 2
Download as PPTX, PDF0 likes54 views
The document provides an overview of exploratory testing in an agile context, defining it as a simultaneous process of learning and testing. It emphasizes the importance of understanding system nouns and verbs, creating context diagrams, and utilizing heuristics to guide exploration. Additionally, it introduces the concept of charters for focused testing sessions and outlines the structure of session reports.
Exploratory testing part 2
- 1. Exploratory Testing, Part 2 Where to go from here Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 2. But first, a little game “Beat the Best Practice” Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 3. Place the oxygen mask on yourself first, before helping small children or others who may need your assistance Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 4. Always wear your seat belt Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 5. You should not smoke Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 6. Exploratory Testing Defined Exploratory Testing involves simultaneously learning about the software under test while designing and executing tests, using feedback from the last test to inform the next. It is NOT: Do random stuff and see what happens, or keyboard banging Unstructured Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 7. Nouns and Verbs Collect a list of nouns and verbs applicable to your current SUT The nouns of the system are the things that it manipulates, manages, and interacts with. The verbs of the system are the actions you can take using the system, or that the system can take. Example: A Shopping Cart Nouns: carts, registered users, items, quantities, prices, discounts, coupons, payment methods, and addresses Verbs: checkout, update, remove, abandon, save, login, logout. Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 8. Alternate Paths As you explore the system noting nouns and verbs, you’re likely to notice that there is more than one way to do things. Whenever you notice that there are multiple ways to get to the same result, make a note. You’ll want to vary these paths as you explore. Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 9. Context Diagrams Software lives within an entire ecosystem. It runs on an operating system, stores data on the file system or in a database, and integrates with other software, like payment gateways for credit card processing, SMTP servers for email, and internal systems like sales reporting, inventory control, and warehouse/shipping applications. As you begin exploring, create a simple context diagram that shows external resources or dependencies, like the one to the left. Over time, when you learn more about the system and the context within which it lives, add more details. Every dependent system provides additional opportunities for exploration. Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 10. Variables Things you can vary More specifically, a variable in testing is anything that you can change, or cause to be changed, via external interfaces (like the UI or the file system), that might affect the behavior of the system Sometimes variables are obviously changeable things like the value in a field on a form. Sometimes they’re obvious, but not intended to be changed directly, like the key/value pairs in a URL string for a web-based application. Sometimes they’re subtle things that can only be controlled indirectly, like the number of users logged in at any given time or the number of results returned by a search. Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 11. Heuristics You’ve started exploring the system. You have the general shape of it. You understand the context in which it lives, the kind of data it manipulates, the actions you can take. You may have some ideas about sequences and configurations that might be interesting. Heuristics: test ideas that are general enough to be applicable across a wide range of domains and technologies. Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 12. Common Heuristics File System Interactions Network Variations CRUD: Create, Read, Update, Delete Position: Beginning, Middle, End Count: 0, 1, Many Interruptions States and Transitions Time: Before, During, After Combining Heuristics Randomizing Noun and Verb Combinations Following the Data Personas Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 13. Test Heuristics Cheat Sheet http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 14. Executing with Charters Traditional scripted testing is documentation-centric with written Test Plans, Test Strategies, Test Cases, and Test Procedures. Exploratory Testing involves far less documentation. We don’t document each and every test case. Instead, we write charters: simple statements of the information that we hope to discover through exploration. Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 15. Charters Explore area With resources, constraints, tools, etc. To discover information Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 16. Some charters are broad: “Explore the system with typical usage scenarios to discover how it works.” Some are narrow: “Explore the File Import feature with various invalid file formats to discover if there are any circumstances under which the error handling does not give a reasonable response to an invalid file.” Some are about valid usage scenarios with representative user personas: “Explore browsing and shopping with a non-technical user perspective to discover how easy or hard it is to buy items.” Others are about misusage scenarios: “Explore the shopping cart feature with a tool to perform http POST requests to discover if there’s any way to get stuff for free.” We might express charters a little differently, and that’s fine. The purpose of the charter is to provide a focus to our explorations. Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 17. Pulling it all together What charters can you think of based on the context list we talked about last week and your experience with our system? Explore eClinicalOS existing code with all current resources to discover any impacts from the new code Explore eClinicalOS with all current resources to discover if there has been an impact on system stability from the new code Explore eClinicalOS with all current resources to discover if there has been an impact on performance (including under high load) Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 18. Sessions Sessions are specific, timeboxed, uninterrupted During a session, if we notice that we’re going off on a tangent, we use the charter to remind us what we’re supposed to be investigating. We might make a note of another charter to investigate later, but we make a point of staying on charter for now. Generally sessions are somewhere between 45 minutes and 2 hours Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 19. Session Reports Session charter (includes a mission statement, and areas to be tested) Tester name(s) Date and time started Data files Test notes Issues Bugs Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
- 20. For more detailed information Content from Elisabeth Hendrickson, Exploratory Testing in an Agile Context, 2009
Editor's Notes
- #11: Consider for example the Therac-25 case that Nancy Leveson describes in her book Safeware. The Therac-25 was a radiation therapy machine that overdosed patients with radiation, killing them. The story dates back to the 1980’s, but it’s still relevant today. According to Nancy Leveson, in at least one case, the malfunction that caused a death could be traced back to the medical technician’s entering and then editing the treatment data in under 8 seconds. That’s the time it took the magnetic locks to engage. Notice two key subtle variables, or things that a user could change: the speed of input and the sequence of actions Further, Leveson found that every 256th time the setup routine ran, it bypassed an important safety check. That’s yet another subtle variable: the number of times the setup routine ran.