Effective Test Estimation
- 1. W7 Test Management 5/10/17 13:45 Effective Test Estimation Presented by: Shyam Sunder Sidra Medical and Research Center Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ info@techwell.com -‐ http://www.starwest.techwell.com/
- 2. Shyam Sunder A PMP® Certified Project Manager with 19 years of IT experience, Shyam Sunder is currently working at Sidra Medical & Research Center (Qatar) and has worked in various reputed organizations like Dell, United Health Group, HCL etc. previously. Shyam's strengths are in the areas of Project Management, Software Quality, and Client relationship management. He has been actively involved in IV&V along with his IT delivery projects. With his consistent track record of successful product introduction and implementation, Shyam is productive both as an individual contributor and project manager. He regularly contributes papers, presentations, and workshops to IT forums. Contact Shyam on LinkedIn and at shyamconf@gmail.com.
- 3. Test Estimation SHYAM SUNDER - PMP, CSM, 6σGB Sidra Medical & Research Center shyamconf@gmail.com
- 4. Agenda v Software Test Estimation v How do we do Estimations ? v Types of Test Estimations v Software Test Estimation - Overview v Issues while Estimating v Software Test Estimation - Techniques v Conclusion
- 5. Software Test Estimation v Estimation is a prediction based on probabilistic assignments v Software estimation is a continuous process, which should be followed and used through out the project life cycle v Effective software estimation helps track and control cost/effort overruns v Estimations covers following broad areas Ø Estimate size Ø Estimate cost & effort Ø Determine the schedule Ø Assess risks
- 6. How Can We do Test Estimations? Test Estimates Test Estimation Model Test team Product Exp. Test Capability … Quality Test Specifications Test Size / Complexity … Test Platform Test Environment Test Tools …
- 7. Types of Test Estimation Manual Test Execution Effort Estimation The specifications of a given test suite 23.5 man-hours Estimated test execution effort Tools, Experience, ... Test Automation Effort Estimation The specifications of a given test suite APIs, Experience, ... 23.5 man-hours Estimated test automation effort Test Coverage x Execution Effort Analysis The specifications of a given test suite Maximum Effort (optional) Quality Criteria Tests sorted to maximize coverage with minimal effort … Tests to not execute Cost-benefit Analysis to Prioritize Tests for Automation The specifications of a given test suite APIs, Experience, ... Tools, Experience, ... Tests to execute manually Frequency of execution Tests to automate …
- 8. Software Test Estimation - Overview Fine tune the estimation process Re-Estimate Estimate Test Size Estimate Cost & Effort Determine Schedule Track & Compare Actual / Estimated Budget Baseline Data Release Roadmap Type of testing Application size INPUTS
- 9. Why Good Estimation is Needed and Why it Fails? v Why good estimates are important? Ø Testing is often blamed for late delivery Ø Testing time is ”squeezed” Ø It promotes early risk assessment v Reasons for poor estimates? Ø Incomplete/ambiguous requirements Ø New technology Ø Skills (Testers & developers) Ø Project delays Ø Poor environments
- 10. Issues While Estimating v Major Issues Ø Estimation method / Process Ø Environment & Tools Ø Testing to be performed Ø Complexity of the application under tests v Minor Issues Ø Test Resource Ø Other factors
- 11. Issues While Estimating - Process v Requirement Stability v Change Requests v Finalize types of testing v Follow the test process v Non-availability of test cases and test data v Timely reviews of the test cases and other artifacts v Coordination with various teams/modules/interfaces v Scope changes
- 12. Issues While Estimating – Environment & Tools v Separate Environment v Environment not similar to the deployment environment. v Downtime of the environment available during testing v Availability of test management tools v Availability of test automation tools
- 13. Issues While Estimating – Environment & Tools v Separate Environment v Environment not similar to the deployment environment. v Downtime of the environment available during testing v Availability of test management tools v Availability of test automation tools
- 14. Issues While Estimating – Testing Resources v Management commitment towards completion and following the test life cycle v Common and realistic expectation toward the testing goal from all the stakeholder in project v Availability of key resources v Application knowledge among the test team v Connect / Attitude between the development and testing team v Correct resolution on the defect fixes v Clear Communication
- 15. Issues While Estimating – Others factors v Complexity of the application under tests v Incorrect assumptions during estimation v Ownership of testing, Independent Development Vendor Customer v Timeline assigned for testing v Development timeline and release of code for testing v Availability of correct test data during test execution v Geographical location of testing team involved v Not doing periodic re-estimation
- 16. Software Test Estimation - Techniques v Simple Medium Complex (SMC) Method v Top Down v Bottom Up v Test Point Analysis (TPA)
- 17. Software Test Estimation – SMC Method This model will consider the test functions / test conditions and their complexities (Simple, Medium, Complex) as the basis for estimation and the effort involved for the following test activities can be estimated using this model. Following test activities could be covered • Test Initiation • Test Planning & Design • Test Execution • Test Closure activities
- 18. SMC Method( Contd.. ) Test Initiation : Effort estimate for the following activities can be done using SMC model under Initiation Phase : v Knowledge Transfer , Application Familiarity, Requirements Analysis. Test Planning and Design: Effort estimate for the following activities can be done using SMC model under Test Planning & Design Phase: v Test Plan ; Preparation of Scenarios, Test Cases, Test Data. v Preparation of Execution Plan v Test Ware Re-work & Reviews & Prepare and Review of Zero-day checklist Test Execution : Effort estimate for the following activities can be done using SMC model under Test Execution Phase : v Creation of test bed & Test Execution ;Review of Defects & Update the Defect/Execution report Test Closure : Effort estimate for the following activities can be done using SMC model under test closure Phase: v Closure Metrics Preparation ,Closure meeting v Archive project data (Project Closure Activities)
- 19. Software Test Estimation – Top Down Method v The Overall effort estimate for the project is determined first in FP or Line of code method. v The effort for various activities are determined. v The method is more applicable to Ø Early cost estimate (High level estimate) Ø When only global properties are known v The estimation procedure is as follows: Ø Get the total size in FP Ø Define the lower level project test component. Ø Based on experience and productivity data from previous projects, obtain the effort estimate § Overall effort estimate = productivity *size
- 20. Software Test Estimation – Bottom Up Method v It is also known as “divide and conquer” technique v It is hierarchical decomposition of the test effort into stages, activities and tasks. Ø Planning Ø Test environment & configuration Ø Test case creation Ø Test execution v Again decompose the above activity in smaller packages which can be estimated in short period of time. v Estimate the total effort by understanding the duration and effort of each activity.
- 21. Dynamic Test Points Static Test Points Environment Factor Total Test Points Productivity Factor Primary Test Hours Total Test Hours Control Factor 1 2 4 3 5 Software Test Estimation –Test Point Analysis
- 22. Test Point Analysis(TPA) – Contd.. Test Point Analysis can be used to objectively prepare an estimate for black box testing (excluding performance testing). Test Case Point Analysis methodology is based on Test Case Points. Test Case Point is a Verification Point used to verify that the value on AUT matches with the expected value. This O/p value can be I/p data for other verification points. v Test Points analysis is a method of estimation, which helps independent testing projects estimate effort required for different phases of testing. v TPA is concerned only with the measurable quality characteristics which fall within the scope of acceptance testing and/or system testing. v TPA is in principle analyst-independent. In other words, if two different people performed a TPA of the same information system, they should obtain the same result. v TPA depends on the availability of a function point count v For TPA purposes, the test team's matter knowledge is not treated as a variable which affects the amount of work involved in the tests v More Suitable for projects which have more planned releases and not adhoc
- 23. Test Point Analysis(TPA) – Contd.. v Experience with the TPA technique suggests that the following percentages are generally appropriate: Test Phase % Test Scripting Preparation (includes Functional Understanding) 10 Specification (includes Test Conditions, Test Data identification, Test Script preparation) 40 Test Execution Test Script Execution and Defect Management (includes Smoke, System, Integration, End to End and Regression Test) 45 Completion (includes UAT) 5
- 24. The estimation technique guidelines explained in the earlier sections can be enhanced to cover the various environmental factors. Pilot these guidelines for couple of projects in your organization and compare the estimated effort and actual effort. As we get proficient with its implementation we will find that Estimated and Actual efforts are getting closer which will result in better execution of the testing projects. Conclusion
- 25. Please feel free to contact me at eMail : shyamconf@gmail.com Twitter : shyam_sunder_30 Linkedin : sundershyam30