Case study dre using 7 qc tools for problem solving
- 1. CASE STUDY Defect Removal Efficiency This case study demonstrates the steps of problem solving using 7 basic quality control tools to solve the problem By Arti Bhatia
- 2. STEPS IN PROBLEM SOLVING 1 2 3 4 5 6 Measure Past Data & Current State Identify Problem Analyse Problem Implement Solution Monitor Solution Control By Arti Bhatia
- 3. MODEL FOR SYSTEMATIC USAGE OF "BASIC QUALITY TOOLS" By Arti Bhatia
- 4. CASE STUDY: PROBLEM DESCRIPTION: CURRENT SCENARIO Suppose you are improving on quality aspects of projects being delivered to improve customer satisfaction. The prime goal is delivering projects with minimum defects. Use 7 QC tools to understand current situation, find root causes, improve requested areas and finally show process control. Current Scenario: A group of data analytics team was deployed on collecting defect related data and following observed things were presented to Higher Management for Month of Mar-19. Phase Wise distribution of Defects Defect Type Review Testing External Testing Phases Requirement Development Design Coding Unit Testing Functional testing System Testing Third Party UAT Mod A 1 5 1 - 4 1 20 25 Mod B - 6 - 1 6 2 19 20 Mod C - 2 2 2 - - 13 15 Mod D 9 - 0 - 2 1 18 20 Mod E 2 1 - 3 5 - 10 15 Mod F - 4 1 7 - - 10 15 Mod G 5 - 1 1 9 1 27 30 Mod H 6 2 - - 3 1 12 15 By Arti Bhatia
- 5. CASE STUDY: PROBLEM DESCRIPTION: CURRENT SCENARIO Defect Trends Data: For period from Jan 17 – Mar 19. As shown in table below: Month Defects Reported by Internal Teams Defects Reported by Third Party Team Defects Reported by UAT teams Jan-17 28 60 70 Feb-17 192 57 65 Mar-17 61 23 42 Apr-17 73 45 34 May-17 158 77 61 Jun-17 131 21 25 Jul-17 182 11 15 Aug-17 104 56 56 Sep-17 48 19 21 Oct-17 99 17 20 Nov-17 161 38 38 Dec-17 146 72 72 Jan-18 60 192 200 Feb-18 57 161 201 Mar-18 23 251 202 Apr-18 45 158 155 May-18 77 131 132 Jun-18 21 182 180 Jul-18 11 104 120 Aug-18 56 148 148 Sep-18 19 99 278 Oct-18 17 161 123 Nov-18 38 146 259 Dec-18 72 60 160 Jan-19 34 212 212 Feb-19 85 85 185 By Arti Bhatia
- 6. CASE STUDY: IDENTIFY PROBLEM: CONTROL CHARTS By Arti Bhatia
- 7. CASE STUDY: IDENTIFY PROBLEM: SCATTER DIAGRAMS By Arti Bhatia
- 8. CASE STUDY: IDENTIFY PROBLEM: LINE CHART Decreasing DRE (Defect Removal Efficiency) of internal testing team By Arti Bhatia
- 9. CASE STUDY: IDENTIFY PROBLEM: CONCLUSIONS A group of data analytics team came up with following observations 1) Summary table of Min, Max and average no. of defects reported between Jan-17 to Mar 19 Data Attributes Defects Reported by Internal Teams Defects Reported by Third party Teams Defects Reported by UAT Teams Max. No. of Defects Reported 192 251 302 Min. No. of Defects Reported 11 11 15 Average No. of Defects Reported 76 100 125 2) The control and line charts (DRE) charts from the defect data shows following observations post Dec 17 a) No. of internal defects Reported were less b) No of Third party reported defects took steep jump c) No. of Defects reported in UAT also took a steep jump d) The trends of DRE of third party testing team are not so competitive hence no gain achieved out of them. 3) Conclusions From charts and summary table The internal team was some what capable to deliver with less defects before Dec 17 Still average no. of defects found in the period were never higher than the external defects found i.e. defect removal efficiency of internal team was low. Hence a root cause analysis is to be done so as to find the causes of delivery projects with more no. of defects. By Arti Bhatia
- 10. CASE STUDY: IDENTIFY PROBLEM: ROOT CAUSE ANALYSIS Problem Statement: High no. of defects reported by customer. High no. of defects reported by customer. Understanding of Requirements Unstructured Internal Review & Testing Process Poort Third party Testing Skills of Resources on testing & review Late Detection of Defects By Arti Bhatia
- 11. CASE STUDY: ANALYSE PROBLEM: CONCLUSIONS The above possible causes were further analyzed using pereto to get major potential causes for the defects. Defect Distribution based on Causes of Defects Cause of Defects No of Defects % Of Defects Internal review and Testing process 120 31% Late detection of defects 106 28% Third party testing 77 20% Skills of resources on testing and review 54 14% Understanding of Requirements 24 6% 120 106 77 54 24 0 20 40 60 80 100 120 140 Internal review and Testing process Late detection of defects Third party testing Skills of resources on testing and review Understanding of Requirments No. Of Defects Causes of Defects Graph 2: Case Study: Example for Pareto 24 120 106 77 54 0 20 40 60 80 100 120 140 Understanding of Requirments Internal review and Testing process Late detection of defects Third party testing Skills of resources on testing and review No. Of Defects Causes of Defects Graph 1: Case Study: Example for Histogram By Arti Bhatia
- 12. CASE STUDY: ANALYSE PROBLEM: CONCLUSIONS Majority of defects were (80%) contributed because of following: Cause of Defects No of Defects % Of Defects Unstructured Internal review and Testing process 120 31% Late detection of defects 106 28% Poor Third party testing 77 20% Hence Improving in these areas will reduce the reported defects by 80 % By Arti Bhatia
- 13. CASE STUDY: ANALYSE PROBLEM: CONCLUSIONS Further root cause analysis was done on potential causes to get real problems High no. of defects reported by customer. Internal Review & Testing Process Third party Testing Late Detection of Defects No unit testing done Reviews of modules missing Review defects missing for most of the modules Testing defects missing for most of modules High defect leakage Defects reported were tested with incomplete requirements document Dedicated methods for review and testing missing Missing check lists No condition on DRE in contract agreement hence high leakage of defects and poor DRE By Arti Bhatia
- 14. CASE STUDY: ANALYSE PROBLEM: CONCLUSIONS Cause of Defects No of Defects % Of Defects Internal review and Testing process 120 31% Late detection of defects 106 28% Third party testing 77 20% The real problems clearly shows that • There is a strong need of improvement in testing & review processes and there awareness. • Also Laidback behavior of third party testing team needs to be handled • Proper documentations to be provided to third party team for testing. • Formal reports to be demanded from them. The SEPG team came up with solutions and implementation strategy as follows • Flowcharts and check points (check sheets) for testers and reviewers • Improvement in testing and review processes • A methodology to collect and report defects using automation tool • To have contractual obligation for third party testing in case of Low DRE from them. Based on Cause and effect diagram a team from SEPG was allocated for improving the processes and awareness on them. By Arti Bhatia
- 15. CASE STUDY: SOLUTION TO PROBLEM: FLOWCHARTS By Arti Bhatia
- 16. CASE STUDY: SOLUTION TO PROBLEM: CHECK SHEETS Verification S.No. Checkpoints Compliance Level Score 1 Are Review/ Verifications procedures and qualification Criteria established? Fully Compliant 2 2 Is the type of review identified? Fully Compliant 2 3 Are the reviewer(s) identified? Fully Compliant 2 4 Are the artifacts available for review? Fully Compliant 2 5 Are the identified work products reviewed/ verified as per plans relevant? Fully Compliant 2 6 Are all review observations tracked and or closed as applicable? Fully Compliant 2 Validation (SQT) S.No. Checkpoints Compliance Level Score 1 Is Validation Plan for Product included in IPMP or maintained separately? Not In Phase 0 Internal Testing 2 Are work products or product components to be tested/ validated identified? Not In Phase 0 3 Are project team members for conduct of validation nominated? Not In Phase 0 4 Is validation environment established? Not In Phase 0 5 Has the criteria and test cases for evaluating compliance been established? Not In Phase 0 6 Is internal testing conducted as per test plan? Not In Phase 0 7 Are all issues/ observations recorded, tracked and fix for closure by relevant stakeholders? Not In Phase 0 8 Is the test summary report prepared and shared as necessary? Not In Phase 0 External Testing – By Third Party 9 Are work products to be verified and validated by IV&V agency identified? (Offering Group) Not In Phase 0 10 Are test cases and qualifying criteria for IV&V testing established, reviewed and approved? Not In Phase 0 11 Is the IV&V environment ready? Not In Phase 0 12 Has the IV&V been conducted and test results, observations and comments recorded and communicated to relevant stakeholders for closure? Not In Phase 0 13 Are all observations and comments being monitored and tracked for closure? (Offering Group) Not In Phase 0 14 Is the IV&V completed and signed off? Not In Phase 0 15 Is the test summary report prepared and shared as necessary? Not In Phase 0 Acceptance Trials/ User Trials 22 Is the product and team ready as per promulgated Trial Directives, if applicable? Not In Phase 0 23 Have the concern agencies been intimated regarding readiness for acceptance testing? Not In Phase 0 24 Is the acceptance testing completed and observations, if any, recorded and forwarded to stakeholders? Not In Phase 0 25 Are the observations fixed and tracked to closure by relevant stakeholders? Not In Phase 0 26 Are the acceptance trials completed and signed off? Not In Phase 0 By Arti Bhatia
- 17. CASE STUDY: IMPROVE (MONITOR): CONTROL CHARTS By Arti Bhatia
- 18. CASE STUDY: IMPROVE (MONITOR): CONTROL CHARTS By Arti Bhatia
- 19. CASE STUDY:CONTROL: CONTROL CHARTS Once the solution is implemented the following things needs to be taken care of: 1. The implemented solution shall be closely monitored 2. Any deviation in expected output shall be subjected to escalation for preventive and/or corrective actions 3. The controlled solution can be subjected to further improvement by setting continual improvement goal. Example in our Case Study : The DRE of internal testing team which was Fluctuating between 10% to 88 % in period Jan-17 to Mar -19 Has been improved to lie between 73% to 87% in period of Apr-19 to Sep -19 To further control and improve the situation A new goal to achieve DRE of 92% can be set and plan for same can be put up. By Arti Bhatia
- 20. By Arti Bhatia