Test Plan Simplicity
Download as PPTX, PDF2 likes1,815 views
This document discusses simplifying test plans by removing unnecessary information and keeping them dynamic. It recommends including only essential information like test ownership, the system configuration under test, definition of done, identified risks, test activities, and a dynamic test schedule. The test plan should evolve continuously through a self-learning loop to improve test scope based on lessons learned. Static information can be moved to other documents to keep the test plan focused on guiding the test project.
![Introduction
•
In this presentation I will use James Bach’s definition of
a test plan [1]
•
Test Plan: the set of ideas that guide a test project
•
Test Strategy: the set of ideas that guide test design
•
Test Logistics: the set of ideas that guide the application of
resources to fulfill a test strategy
•
Every artifact created must add value
•
The process of creating a test plan adds value, but the
document itself is of limited value
•
2
2013-11-12
PA1
The value of a test plan is the ability to communicate the
set of ideas that guides the test project to different
stakeholders
Confidential](https://image.slidesharecdn.com/simplicityandtestplans-131113023217-phpapp01/85/Test-Plan-Simplicity-2-320.jpg)
![Simplicity
•
•
We also want to limit the amount of effort we spend on
creating the test plan, and minimize the amount of
unnecessary artifacts created
•
2013-11-12
There is a value in keeping the test plan simple which
relates to the burden which the test plan puts on
someone trying to explain or understand it
•
3
Simplicity is the state or quality of being simple [2]
On Google and Microsoft there is a concept of a 10
minute test plan, which should in accordance with the
name only take a short period of time to create [3]
PA1
Confidential](https://image.slidesharecdn.com/simplicityandtestplans-131113023217-phpapp01/85/Test-Plan-Simplicity-3-320.jpg)
![Definition of Done
•
It is necessary to have a Definition of Done [4] for
different phases of the test project
•
This is used to define when testing stops as implied by
the name
•
Definition of Done should include:
•
•
2013-11-12
PA1
Expected Risk Level
•
10
Expected Test Coverage
•
•
Expected System Quality
Formal Documentation Needed
Definition of Done can change over time dependant on
stakeholder needs
Confidential](https://image.slidesharecdn.com/simplicityandtestplans-131113023217-phpapp01/85/Test-Plan-Simplicity-10-320.jpg)
![Risk Quantification
•
The Risk Matrix needs to be filled quantifiable risks
•
There are many different models available
•
Severity - Occurrence - Detection [5] is one
•
Low risk, Medium risk, High risk, is another
•
•
12
2013-11-12
If a system has many configurations, each
configuration needs a column in the risk matrix
Risks most likely also need to be categorized in a good
way to make the matrix usable
PA1
Confidential](https://image.slidesharecdn.com/simplicityandtestplans-131113023217-phpapp01/85/Test-Plan-Simplicity-12-320.jpg)
![Test Schedule
•
•
14
2013-11-12
A schedule or plan of when each test activity should be
executed in time
The schedule should be dynamic [6] and based on input
from what is happening in the project, and what has
happened during previous test activities
PA1
Confidential](https://image.slidesharecdn.com/simplicityandtestplans-131113023217-phpapp01/85/Test-Plan-Simplicity-14-320.jpg)
![References
[1] A question about test strategy
http://www.satisfice.com/blog/archives/63
[2] Simplicity
http://en.wikipedia.org/wiki/Simplicity
[3] 10 Minute Test Plan
http://googletesting.blogspot.se/2011/09/10-minute-test-plan.html
[4] Definition of Done
https://www.scrum.org/Resources/Scrum-Glossary/Definition-of-Done
[5] Severity, Occurrence, and Detection Criteria for Process FMEA
http://www.fmeainfocentre.com/guides/ProcessPktNewRatings.pdf
[6] Dynamic Test Plans
http://www.slideshare.net/JohanHoberg/dynamic-test-plans
17
2013-11-12
PA1
Confidential](https://image.slidesharecdn.com/simplicityandtestplans-131113023217-phpapp01/85/Test-Plan-Simplicity-17-320.jpg)
Test Plan Simplicity
- 2. Introduction • In this presentation I will use James Bach’s definition of a test plan [1] • Test Plan: the set of ideas that guide a test project • Test Strategy: the set of ideas that guide test design • Test Logistics: the set of ideas that guide the application of resources to fulfill a test strategy • Every artifact created must add value • The process of creating a test plan adds value, but the document itself is of limited value • 2 2013-11-12 PA1 The value of a test plan is the ability to communicate the set of ideas that guides the test project to different stakeholders Confidential
- 3. Simplicity • • We also want to limit the amount of effort we spend on creating the test plan, and minimize the amount of unnecessary artifacts created • 2013-11-12 There is a value in keeping the test plan simple which relates to the burden which the test plan puts on someone trying to explain or understand it • 3 Simplicity is the state or quality of being simple [2] On Google and Microsoft there is a concept of a 10 minute test plan, which should in accordance with the name only take a short period of time to create [3] PA1 Confidential
- 4. Simplicity Cost of Simplicity Value of Simplicity Complicated 4 2013-11-12 PA1 Simple Confidential
- 5. Dynamic Test Plan • • 2013-11-12 Static information can be moved from a test plan to a Test Strategy or a Test Logistics document • 5 The key to reducing the size of the test plan is to remove anything that cannot change over time Anything that does not add value to any stakeholder should also be removed PA1 Confidential
- 6. Content of a Test Plan • • Definition of Done • System Risk Matrix • Test Activities • Test Schedule • 2013-11-12 System Configuration Under Test • 6 Test Ownership Self-Learning Loop (SLL) PA1 Confidential
- 7. Overview Ownership - Test Configuration - Definition of Done - Activity Description Identify Risks 7 2013-11-12 PA1 Plan Test Activities Self-Learning Loop Confidential
- 8. Test Ownership • Which person is responsible for performing what tests? • This could be done on a overview scope level, for different test charters, or for specific test cases • A test ownership for a specific person or group could be defined in the following ways: • • Performance • PA1 All or specific charters related to performance • 2013-11-12 All performance tests • 8 250 specific performance test cases Running performance tests during a specific time period or project phase Confidential
- 9. System Configuration Under Test • Most systems can be configured in many different ways • It is necessary to describe the configuration under test in order to • • 2013-11-12 PA1 Divide test ownership between different configurations • 9 Write bug reports • • Reproduce tests Let all testers know what to test on This can change over time depending on the configurability of the system Confidential
- 10. Definition of Done • It is necessary to have a Definition of Done [4] for different phases of the test project • This is used to define when testing stops as implied by the name • Definition of Done should include: • • 2013-11-12 PA1 Expected Risk Level • 10 Expected Test Coverage • • Expected System Quality Formal Documentation Needed Definition of Done can change over time dependant on stakeholder needs Confidential
- 11. System Risk Matrix • What risks threaten the system from reaching the definition of done? • These risks change continuously over the life cycle of the project • Risks can include (but are not limited to): • • Hardware delta between configurations • Unknown system dependencies • Unknown environment dependencies • PA1 Software delta between configurations • 2013-11-12 New hardware builds • 11 Software changes Changed stakeholder focus Confidential
- 12. Risk Quantification • The Risk Matrix needs to be filled quantifiable risks • There are many different models available • Severity - Occurrence - Detection [5] is one • Low risk, Medium risk, High risk, is another • • 12 2013-11-12 If a system has many configurations, each configuration needs a column in the risk matrix Risks most likely also need to be categorized in a good way to make the matrix usable PA1 Confidential
- 13. Test Activities • • 2013-11-12 The scope of these test activities should not be static, since it is seldom efficient to run static activities • 13 All different test activities need to be explained in the test plan Different activities can be added and removed during the life cycle of the project PA1 Confidential
- 14. Test Schedule • • 14 2013-11-12 A schedule or plan of when each test activity should be executed in time The schedule should be dynamic [6] and based on input from what is happening in the project, and what has happened during previous test activities PA1 Confidential
- 15. Self-Learning Loop (SLL) • • 2013-11-12 Continuous improvement of gaps in test scope • 15 It is critical to describe how the test plan will evolve over the project life cycle, and then continuously document what we have learned from this feedback loop Continuous improvement of understanding of system dependencies PA1 Confidential
- 16. Conclusion • • 2013-11-12 Test plans should be dynamic – static parts should be included in other documents which can be reused between projects, or not documented at all • 16 Test plans should add value and not include any waste These 7 bullets help include critical parts in the test plan, and not introduce unnecessary waste PA1 Confidential
- 17. References [1] A question about test strategy http://www.satisfice.com/blog/archives/63 [2] Simplicity http://en.wikipedia.org/wiki/Simplicity [3] 10 Minute Test Plan http://googletesting.blogspot.se/2011/09/10-minute-test-plan.html [4] Definition of Done https://www.scrum.org/Resources/Scrum-Glossary/Definition-of-Done [5] Severity, Occurrence, and Detection Criteria for Process FMEA http://www.fmeainfocentre.com/guides/ProcessPktNewRatings.pdf [6] Dynamic Test Plans http://www.slideshare.net/JohanHoberg/dynamic-test-plans 17 2013-11-12 PA1 Confidential