Paula O' Grady - Prioritising tests? - Use Your Gut Instinct
Download as PPT, PDF0 likes770 views
The document discusses prioritizing software testing amidst time constraints, emphasizing the need for innovative methods alongside traditional techniques. It introduces a betting approach to gauge where bugs might occur, encouraging participation from non-technical team members, while also highlighting the advantages and disadvantages of this method. The conclusion suggests integrating these modern techniques into test planning to better identify unexpected bugs in production.
Paula O' Grady - Prioritising tests? - Use Your Gut Instinct
- 2. Prioritising Tests - Why? You never have enough time to do all the testing you want/need to do. Allocated testing time may ‘shrink’. How to decide which tests take precedence? Attach a severity or priority, or a calculation of the two. Other labour-intensive methods. Limitations with this approach in my experience!
- 3. Prioritising Tests – Issues? Ironically, the traditional prioritising techniques you use may deplete available time even more. Like buying an expensive wallet and not having enough money left to put into it! Perhaps we need to examine the methods we use to prioritise tests to maximise efficiency. In addition, alternative techniques may provide insight, and a new viewpoint. Historically, the best ideas have come from the most unlikely sources.
- 4. Possible Solutions Lateral-thinking approach. As with most ‘radical’ methods, they work best when used in tandem with the tried-and-tested traditional methods. Betting & weighting approaches involving teams guessing where most bugs will be found. Dynamic Betting Exchanges. Split-Second Decision Making and Rapid-Cognition. Gut-Instinct.
- 5. Betting Approach Divide the product into functional/test areas – this is the difficult part in my experience. Give each person in the group a virtual €100 to ‘bet’, using as many denominations as they wish. The non-technical team members were especially encouraged to provide gut-instinct feedback, independent of ‘developmenttest’ thinking. In theory, participants can change their bets on the fly and as regularly as they wish – exchange aspect. No need for sophisticated tools.
- 6. Betting Approach – Case Study Collate data: JW FM EMG DMW PD SOS POG CK CON DMC AK RR MM AC Total % 1Create Webs 10 2 5 10 2 2.6 2Alerts and Notifications 10 2 1.1 3Documentation 5 10 15 10 10 5 15 5 6.8 4Export Report and Generate Report 5 3 2 10 5 4 2.6 5Globalisation & Localisation 5 5 5 1.4 6Kit – Install and Uninstall 5 10 10 5 5 3.2 7Kit – Upgrade 5 15 10 10 5 5 4.5 8Project Schedule List 10 33 40 35 10 15 20 15 10 17.0 9All Other Lists and Libraries 0.0 10Usability 5 0.5 11Security 10 2 5 1.5 12Load and Performance 2 10 10 5 2.1 2.6 13BrightWork Reporter & Resource Reporting* 50 5 30 10 30 30 15 20 10 27.9 20.7 14Gantt Web Part 1 3 2 25 2.8 15*Chart Web Part 5 10 10 30 15 20 15 8 20 12.1 16Agile Template 1 5 3 2 1.0 17Sure Step Templates 2 5 5 3 2 10 2.4 18All Other Templates 10 20 10 10 5 5.0 19All Other Web Parts 4 0.4 20Choice Indicator Icon Column* 2 5 10 5 1 5 2.5 21Project Calendar* 5 20 30 10 10 2 25 9.2 22Other – Please Specify 0.0
- 7. Betting Approach – Case Study The bets were charted. Scale runs from 0% to 25 %.
- 8. Betting Approach – Case Study Advantages Disadvantages Quick. Fun. Give people a chance to get involved (who otherwise wouldn’t). Less weighted by the thinking of the core technical group. Allows for ‘unexpected’ anomalies to be found. Difficult to define the ‘areas’ to bet on. May not be taken seriously. Cannot be used on its own. People may try to ‘follow the crowd’ and bet safely – this defeats the purpose. Needs management buy-in.
- 9. Rapid Cognition Looking away from the arena of Software Testing. Books that provide insight into how to make effective decisions quickly: Blink – Malcolm Gladwell The Gift of Fear – Gavin deBecker Other Books Not written specifically for a Software Test audience, but the principles and how they apply to decision- making are universal.
- 10. Rapid Cognition What can these books teach us about prioritising tests? The lessons here are universal. In times of stress, decisions need to be made. How firemen, policemen & doctors make life- changing and life-saving decisions. Is there such a thing as ‘Too Much Information’ when making decisions?
- 11. Gut Instinct Does “Gut feeling = Irrational Decision Making”? In testing terms, using gut instinct could be viewed as analogous to Exploratory Test Techniques for software. Not having to explain each small decision and just going with your instinct. Not necessarily being psychic – you may feel a piece of code has a bug in it because of something that happened the day it was being developed. The feeling may not necessarily be fully articulated.
- 12. Conclusions The techniques described in this presentation should be used in addition to traditional methods As Agile methods become more commonplace, we need to look at modernising all activities to do with Test Planning. The bugs that make it into production are generally the ones ‘that we didn’t expect’. Less traditional methods of prioritising may help mitigate this risk.