Improving Defect Yield - a three step approach
1 like920 views
The document describes a three-step approach to improving defect yield in testing: 1. Conduct a potency assessment to determine which types of defects are being targeted by current test cases and if any important types are missing. 2. Perform potential defect type re-targeting to add new test cases to cover additional defect types identified as missing. 3. Enhance existing test cases through potency improvement to ensure they are complete in uncovering defects.
Improving Defect Yield - a three step approach
- 1. Improving defect yield - A three step approach Powered by HBT T Ashok Founder & CEO STAG Software Private Limited in.linkedin.com/in/AshokSTAG Ash_Thiru
- 2. © 2013 STAG Software Private Limited.All rights reserved. Defect Yield 2 Defect yield is the #defects (of good quality) uncovered by the test cases Yield = Outcome/Effort Defect yield = #Defects / # TestCases
- 3. © 2013 STAG Software Private Limited.All rights reserved. Expectation & Challenges 3 Expectation Increasing the yield - a natural expectation. Challenges Defect yield drops/is-low : - when system matures - when system of testing is “weak” - when testing is transitioned to a different team
- 4. © 2013 STAG Software Private Limited.All rights reserved. Why Lower Yield ? 4 Are we wasting effort by testing stable parts? Are we missing doing something else?
- 5. © 2013 STAG Software Private Limited.All rights reserved. The 3-step approach to yield improvement 5 Potency assessment1 Potential defect type re-targeting2 Potency improvement3 What types of defects are being targeted by the test cases? Are the test cases complete/adequate? Are there any other types of defects to be targeted? Add new test cases to cover the new types of defects Enhance existing test cases to ensure that they are complete
- 6. © 2013 STAG Software Private Limited.All rights reserved. Hypothesis Based Testing - HBT 6 System Under Test Cleanliness Criteria Potential Defect Types Test CasesRequirements traceability “what to test” Fault traceability “test for what” should satisfy impeded by
- 7. © 2013 STAG Software Private Limited.All rights reserved. 7 Quality Levels & PDTs - Powered by HBT Input cleanliness Input interface cleanliness Structural integrity Behaviour correctness Environment cleanliness Attributes met Flow correctness Clean Deployment End user value L1 L2 L3 L4 L5 L6 L7 L8 L9 That inputs are handled well PDTs related to Input data correctness That the functional behaviour is correct PDTs related to Functionality That the internal structure is robust PDTs related to Structural aspects That the user interface is clean PDTs related to UI That end-to-end flows work correctly PDTs related to Flow behaviour, Interactions That it does not mess up the environment PDTs related to Resource leaks, Compatibility... That the stated attributes are met PDTs related to Performance, Load, Volume...... That it deploys well in the real environment PDTs related to Compatibility, Migration That user expectations are met PDTs related to User flows, User experience
- 8. © 2013 STAG Software Private Limited.All rights reserved. Potency Framework (For Assessment & Re-target) 8 QL9 End user value End user value test PDT 57-59 QL8 Clean deployment Installation test PDT 54-55 Migration test PDT 56 Migration test QL7 Attributes met LSPS test PDT 44-51 Reliability test PDT 52 Reliability test PDT 52 Security test PDT 53 QL6 Environment cleanliness Good citizen test PDT 39-41 Compatibility test PDT 42-43 Compatibility test QL5 Flow correctness Interaction test PDT 35-38 QL4 Behaviour correctness Functionality test PDT 24-31 Access control test PDT 32-34 Access control test QL3 Structural integrity Structural test PDT 14-23 QL2 Input interface cleanliness API validation test PDT 5-7 GUI validation test PDT 8-13 GUI validation test QL1 Input cleanliness Input validation test PDT 1-4 Input validation test QL - Quality Level PDT - Potential Defect Type PoweredbyHBT
- 9. © 2013 STAG Software Private Limited.All rights reserved. An illustration : A component of a large system 9 Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1 BeforeBeforeBeforeBeforeBefore ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After) #PDT s #TCs#TCs#TCs #Defects '+PDTs '+TCs'+TCs'+TCs #Defects Total '+ve '-ve Total '+ve '-ve L8 0 0 L7 1 1 1 0 1 2 1 1 L6 0 0 L5 0 0 L4 7 1116 976 140 8 7 45 26 19 17 L3 1 135 130 5 0 L2 2 29 26 3 0 L1 3 23 9 14 0 Total 14 1304 1142 162 8 8 47 27 20 17 Increase byIncrease byIncrease byIncrease byIncrease by 57% 4% 2% 12% 213% TC YieldTC Yield 0.01 Defects/TCDefects/TC TC YieldTC Yield 0.36 Defects/TCDefects/TC Potency assessment PDT re-targeting21 3 Potency improvement
- 10. © 2013 STAG Software Private Limited.All rights reserved. An illustration - The entire system 10 Potency assessment PDT re-targeting21 3 Potency improvement Customer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : Summary BeforeBeforeBeforeBeforeBefore ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After) #PDTs #TCs#TCs#TCs #Defects '+PDTs '+TCs'+TCs'+TCs #Defects Total '+ve '-ve Total '+ve '-ve C1 14 1304 1142 162 8 8 47 27 20 17 C2 20 543 353 190 6 8 176 114 62 7 C2 6 741 494 247 19 7 17 12 5 0 Total 40 2588 1989 599 33 23 240 153 87 24 Increase byIncrease byIncrease byIncrease byIncrease by 58% 9% 8% 15% 73% TC YieldTC Yield 0.01 Defects/TCDefects/TC TC YieldTC Yield 0.10 Defects/TCDefects/TC 10x yield improvement
- 11. © 2013 STAG Software Private Limited.All rights reserved. HBT Overview SIX staged purposeful activities, powered by EIGHT disciplines of thinking 11 D1 D2 D4 D3 D5D6 D7 D8 Core Concepts S1 S2 S3S4 S5 S6 SIX Stages of DOING S1: Understand expectations S2: Understand context S3: Formulate hypothesis S4: Devise proof S5:Tooling support S6:Assess & Analyse EIGHT Disciplines of Thinking D1: Business value understanding D2: Defect hypothesis D3: Strategy & Planning D4:Test design D5:Tooling D6:Visibility D7: Execution & Reporting D8:Analysis & Management Uses 32 Core Concepts For Problem solving - Techniques, Principles, Guideline Click here to know more about HBT. http://stagsoftware.com/blog?p=570
- 12. © 2013 STAG Software Private Limited.All rights reserved. HBT is the intellectual property of STAG Software Private Limited. STEMTM is the trademark of STAG Software Private Limited. www.stagsoftware.com @stagsoft blog.stagsoftware.com Connect with us... Improving defect yield - A three step approach Powered by HBT Potency assessment1 Potential defect type re-targeting2 Potency improvement3 Thank you.