![1.6.12 Factors affecting Test Estimation
Productivity Figure: It is based on knowledge and skill of test team members and is
therefore specific to the individual organization. Productivity figure mentioned in these
guidelines needs to be verified for couple of projects before implementation across
GSI.
Environmental Factor: Following environmental factors should be consider for test
effort estimation:
Test Tools: It reflects the extent to which testing is automated, or the extent to which
automation tools are used for testing.
Development Testing: It reflects the extent to which the development testing is down,
a development test plan is available and test team is familiar with the actual test cases
and test results
Test Base: It reflects the quality of system documentation upon which the test under
consideration is to be based.
Test Environment: It reflects the extent to which the test infrastructure in which the
testing is to take place has previously been tried out.
Testware: It reflects the extent to which the tests can be conducted using existing
testware.
Multiple Browsers: Effort estimation for testing on multiple browsers is more then
testing on one browser.
1.7 Conclusion
The estimation technique guidelines explained in the earlier section can be enhanced
to cover the various environmental factors. Pilot these guidelines for couple of projects
in your organization and compare the estimated effort and actual effort. As we get
proficient with its implementation we’ll find that Estimated and Actual efforts are
getting closer which will result in better execution of the testing projects.
I’m also attaching a sample Testing Estimation Sheet created by me and which I’ve
used extensively. This estimation tool has been successfully implemented in my
previous organizations also.
Test Estimation
Tool.xls
[Note: The attached template is just a snapshot of the actual tool.]
1.8 References:
The whole content has been written based upon my past experiences in various
organizations. The views may differ based upon circumstances. Feel free to get back
to me at sundershyam30@yahoo.com in case of any clarifications.
10 | P a g e](https://image.slidesharecdn.com/pmt05-101207044239-phpapp01/85/Pmt-05-10-320.jpg)
Pmt 05
- 1. Aum gam ganapataye namya. Test Estimation Shyam Sunder, PMP Dell Services
- 2. Contents 1.1 Abstract 3 1.2 Audience: 3 1.3 Area of Application: 3 1.4 Issues and Challenges: 3 1.4.1 Issues While Estimating - Process 3 1.4.2 Issues While Estimating – Environment & Tools 3 1.4.3 Issues While Estimating – Testing Resources 3 1.4.4 Issues While Estimating – Others factors 4 1.5 Content 4 1.5.1 There are various issues which creep up while estimating 6 1.5.2 Issues While Estimating – Environment & Tools 6 1.5.3 Issues While Estimating – Testing Resources 6 1.5.4 From the above we have got a basic idea about the facts related to Test estimations. 6 1.6 Now in the following sections I’d be explaining the above mentioned estimation techniques in detail. 7 1.6.1 SMC Method 7 1.6.2 Test Initiation: 7 1.6.3 Test Planning and Design: 7 1.6.4 Test Execution: 7 1.6.5 Test Closure: 8 1.6.6 Top Down Method 8 1.6.7 Bottom Up Method 8 1.6.8 Test Point Analysis(TPA) 9 1.6.9 Details about Test Case Point are as follows: 9 1.6.10 Test Scripts can be defined in following three Complexity Levels: 9 1.6.11 Breakdown between Testing Phases: 9 1.6.12 Factors affecting Test Estimation 10 1.7 Conclusion 10 1.8 References: 10 1.9 About the Author 11 2|Page
- 3. 1.1 Abstract We have executed many projects Large Projects, Small Projects etc. Sometimes we miss our testing deadlines because there is no defined criterion that is used to build our execution test plan. To help avoid such missing our deadlines we have prepared these Test Estimation guidelines. In this paper I present the various Test estimation techniques which will help us in proper execution of the Testing projects and also one estimation model created by me which is being widely used in my present organization. This is a paper submitted under “Program Management Tools” track. 1.2 Audience: Test Management Professionals — Managers, Architects, Test Leads, Software Test Engineers Quality / SEPG Management Professionals — Managers, QA Leads, Software QA Engineers Project Management Professionals — Managers, Leads, Software Engineers 1.3 Area of Application: Testing Projects 1.4 Issues and Challenges: Major Issues : Estimation method / Process, Environment & Tools, Testing to be performed, Complexity of the application under tests Minor Issues : Test Resource, Other factors 1.4.1 Issues While Estimating - Process Requirement Stability, Change Requests, Finalize types of testing ,Follow the test process,Non-availability of test cases and test data, Timely reviews of the test cases and other artifacts, Coordination with various teams/modules/interfaces, Scope changes 1.4.2 Issues While Estimating – Environment & Tools Separate Environment, Environment not similar to the deployment environment, Downtime of the environment available during testing, Availability of test management tools, Availability of test automation tools. 1.4.3 Issues While Estimating – Testing Resources Management commitment towards completion and following the test life cycle, Common and realistic expectation toward the testing goal from all the stakeholder in project, Availability of key resources, Application knowledge among the test team, Connect / Attitude between the development and testing team, Correct resolution on the defect fixes, Clear Communication 3|Page
- 4. 1.4.4 Issues While Estimating – Others factors Complexity of the application under tests, incorrect assumptions during estimation, Ownership of testing, Independent Development Vendor Customer Timeline assigned for testing, Development timeline and release of code for testing, Availability of correct test data during test execution, Geographical location of testing team involved, Not doing periodic re-estimation. 1.5 Content Test Estimation is a prediction based on probabilistic assignments and is a continuous process, which should be followed and used through out the project life cycle. Effective software estimation helps track and control cost/effort overruns. Estimations cover following broad areas Estimate size Estimate cost & effort Determine the schedule Assess risks Now this brings us to a basic question that how can we do Test Estimations? Test Estimations can be done as shown in the Fig 1. Fig. 1 4|Page
- 5. There are various types of Test Estimations as shown in Fig. 2: Fig. 2 Software Test Estimation – Overview as shown in Fig 3 Fig. 3 5|Page
- 6. 1.5.1 There are various issues which creep up while estimating Major Issues Estimation method / Process Environment & Tools Testing to be performed Complexity of the application under tests Minor Issues Test Resource Other factors 1.5.2 Issues While Estimating – Environment & Tools Separate Environment Environment not similar to the deployment environment. Downtime of the environment available during testing Availability of test management tools Availability of test automation tools 1.5.3 Issues While Estimating – Testing Resources Management commitment towards completion and following the test life cycle Common and realistic expectation toward the testing goal from all the stakeholder in project Availability of key resources Application knowledge among the test team Connect / Attitude between the development and testing team Correct resolution on the defect fixes Clear Communication 1.5.4 From the above we have got a basic idea about the facts related to Test estimations. There are various software test estimation techniques: Simple Medium Complex(SMC) Method Top Down Method Bottom Up Method Test Point Analysis(TPA) 6|Page
- 7. 1.6 Now in the following sections I’d be explaining the above mentioned estimation techniques in detail. 1.6.1 SMC Method This model will consider the test functions / test conditions and their Complexities (Simple, Medium, and Complex) as the basis for estimation and the effort involved for the following test activities can be estimated using this model. Following test activities could be covered Test Initiation Test Planning & Design Test Execution Test Closure activities 1.6.2 Test Initiation: Effort estimate for the following activities can be done using SMC model under Initiation Phase: Knowledge Transfer Application Familiarity Requirements Analysis Functional Decomposition 1.6.3 Test Planning and Design: Effort estimate for the following activities can be done using SMC model under Test Planning & Design Phase: Test Plan Preparation of Scenarios, Test Cases, Test Data. Test Case, Test Data Reviews Preparation of Execution Plan Test Ware Re-work & Reviews Prepare and Review of Zero-day checklist 1.6.4 Test Execution: Effort estimate for the following activities can be done using SMC model under Test Execution Phase: Verify zero day check list Creation of test bed Test Execution 7|Page
- 8. Review of Incident logs Update the Incident report 1.6.5 Test Closure: Effort estimate for the following activities can be done using SMC model under test closure Phase: Closure Metrics Preparation Closure meeting Archive project data (Project Closure Activities) Test / Project Management I’m attaching a Sample SMC Sheet which I had used for one of my test projects. SMC.xls 1.6.6 Top Down Method In this method, the Overall effort estimate for the project is determined first in FP or Line of code method. The estimation procedure is as follows: Get the total size in FP Define the lower level project test component. Based on experience and productivity data from previous projects, obtain the effort estimate Overall effort estimate = productivity *size 1.6.7 Bottom Up Method This is also known as “divide and conquer” technique. It is hierarchical decomposition of the test effort into stages, activities and tasks. Planning Test environment & configuration Test case creation Test execution Again decompose the above activity in smaller packages which can be estimated in short period of time. Estimate the total effort by understanding the duration and effort of each activity. 8|Page
- 9. 1.6.8 Test Point Analysis(TPA) Test Point Analysis can be used to objectively prepare an estimate for black box testing (excluding performance testing). Test Case Point Analysis methodology is based on Test Case Points. Test Case Point is a Verification Point used to verify that the value on AUT matches with the expected value. This O/p value can be I/p data for other verification points. Following factors will have influence on number of Test Case Points: Complexity: It relates to the number of conditions in a function. More conditions almost mean more test cases and therefore a greater volume of testing work. Interfacing: The degree of interfacing of a function is determined by number of data sets maintained by a function and the number of other functions, which make use of those data sets. Uniformity: The extent to which the structure of a function allows it to be tested using existing or slightly modified specifications, i.e. the extent to which the information system contain similarly structured functions. 1.6.9 Details about Test Case Point are as follows: Low Complexity Test Case Point: A Test Case Point having 1 to 3 Steps is considered as Low Test Case Point. Medium Complexity Test Case Point: A Test Case Point having 3 to 4 Steps is considered as Medium Test Case Point. Critical Complexity Test Case Point: A Test Case Point having 5 to 6 Steps is considered as Critical Test Case Point. 1.6.10 Test Scripts can be defined in following three Complexity Levels: S. No. Level Definition 1 Critical If a Test Script is having 6 to 8 Test Case Points Or Verification Points. 2 Medium If a Test Script is having 4 to 5 Test Case Points Or Verification Points. 3 Low If a Test Script is having 1 to 3 Test Case Points Or Verification Points. 1.6.11 Breakdown between Testing Phases: % Testing Phase age Preparation (includes Functional Understanding) 10 Test Scripting Specification (includes Test Conditions, Test Data 40 identification, Test Script preparation) Test Script Execution and Defect Management Test (includes Smoke, System, Integration, End to End and 45 Execution Regression Test) Completion (includes UAT) 5 9|Page
- 10. 1.6.12 Factors affecting Test Estimation Productivity Figure: It is based on knowledge and skill of test team members and is therefore specific to the individual organization. Productivity figure mentioned in these guidelines needs to be verified for couple of projects before implementation across GSI. Environmental Factor: Following environmental factors should be consider for test effort estimation: Test Tools: It reflects the extent to which testing is automated, or the extent to which automation tools are used for testing. Development Testing: It reflects the extent to which the development testing is down, a development test plan is available and test team is familiar with the actual test cases and test results Test Base: It reflects the quality of system documentation upon which the test under consideration is to be based. Test Environment: It reflects the extent to which the test infrastructure in which the testing is to take place has previously been tried out. Testware: It reflects the extent to which the tests can be conducted using existing testware. Multiple Browsers: Effort estimation for testing on multiple browsers is more then testing on one browser. 1.7 Conclusion The estimation technique guidelines explained in the earlier section can be enhanced to cover the various environmental factors. Pilot these guidelines for couple of projects in your organization and compare the estimated effort and actual effort. As we get proficient with its implementation we’ll find that Estimated and Actual efforts are getting closer which will result in better execution of the testing projects. I’m also attaching a sample Testing Estimation Sheet created by me and which I’ve used extensively. This estimation tool has been successfully implemented in my previous organizations also. Test Estimation Tool.xls [Note: The attached template is just a snapshot of the actual tool.] 1.8 References: The whole content has been written based upon my past experiences in various organizations. The views may differ based upon circumstances. Feel free to get back to me at sundershyam30@yahoo.com in case of any clarifications. 10 | P a g e
- 11. 1.9 About the Author Shyam Sunder is a PMP® Certified Test Manager. Shyam has got total IT Testing Experience of 12.5 Years and has worked in various reputed organizations like IIS Infotech, HCL Technologies, Sapient etc. Shyam is strong in the area of Test Management, Software Testing areas and Client relationship management. He is well versed in testing areas and has been actively involved in IV&V along with his testing delivery projects. The strong testing background which Shyam inculcated in previous organizations is being put into forte in Sapient which is reaping rich dividends of his testing acumen and expertise. Shyam has a consistent track record of successful product introduction and implementation. And is Productive as both individual contributor and Project Manager. Shyam also possesses excellent communication and relationship-building skills. Shyam has been a regular contributor in Testing Forums like Quality Assurance of India (QAI), Software Testing Conferences (STC), Step-in-Summits etc with his papers, presentations and workshops. Shyam’s recent presentation was in STeP-In Summit Feb’10 and his topic was “Test Estimation” E-mail: sundershyam30@yahoo.com 11 | P a g e