Top 5 pitfalls of software test automatiion
- 2. Why Automate Your Tests At All It’s about 100 times more costlier to fix a defect after the product has gone live, rather than detecting & fixing it during the testing cycle itself!
- 3. How to Avoid Post Productions Defects Q. You got the software out in a hurry but what happens if your clients discover bugs in the software? R. With automated tests you spot & fix bugs early in the development lifecycle itself. Benefits Investment Gain Costs (Both Direct & Indirect Costs_________ = - __________________________ Investment ROI = The general formula for calculating ROI (which is also applicable for test automation tools) is as below:
- 4. Direct Cost ➢ Tool acquisition & evaluation costs ➢ License costs ➢ Tool training of core automation team ➢ Initial configuration of the tool ➢ Application specific costs – Including creation of a test automation framework, test data, creation, maintenance & execution of manual and automated test cases
- 5. Indirect Cost ➢ Risk of undetected failures before the application hits production ➢ Costs attached to these undetected failures Note: Only if the benefits/ gains outweigh the costs & invested amounts do you reap a positive ROI. So what do you think are some of the pitfalls which can prohibit you from enjoying a positive ROI from your Test automation tool?
- 6. Top 5 Pitfalls for Test Automation 1. To execute everything from test planning to execution! 2. To automate all tests! 3. Require extensive coding! 4. Throw up False Positives! 5. You can replace testers!
- 7. Top 5 Pitfalls for Test Automation 1. To execute everything from test planning to execution! • Test Automation Tool doesn’t really investigate bugs and it can quietly slip away unnoticed! • Test Automation Tool just runs a series of yes / no questions to determine the results. • The Magician Tester runs the whole process of specific action/ inspect result as checking not testing. • Any new information isn’t considered or even observed by the tool. • Do not leave the entire testing strategy & execution process to the Test Automation Tool itself, you are surely headed for disaster. • There is a critical need for human intervention here for analyzing the data, decide on doable features, which problems to fix, then to design & redesign tests and execute them until you have the final piece of bug free software.
- 8. Top 5 Pitfalls for Test Automation • Reduses & Saves time in runnning all tests, It’s a Time Saver • Can run mulitiple test with more and more scenarios • Can run parallel tests from different servers too • Can classify the tests and decide which piece of software can be tested manually and which needs to be automated. • In a short span of time, you can run Unit, Functional, Regression, and Performance Tests seamlessly on cloud without having to invest in obsolete hardware. • You have an option to import the manual test cases into the test automation tool itself - it’s an icing on the cake • Both your test & development teams can have a bird’s eye view of the whole test project from start to finish. 2. To automate all tests!
- 9. Top 5 Pitfalls for Test Automation • You need recalculation and someone needs to write a code or an algorithm to check if the software passes or fails this on new requirement. • Testing teams needs to know, how to code; especially if you don’t have enough programmers on board. • Test Automation Tool that does away with the entire need to code itself. Go for a tool with an inbuilt Scriptless automation feature means your testers enter keywords in English; the tool interfaces with the framework & executes the rest. 3. Require extensive coding!
- 10. Top 5 Pitfalls for Test Automation • False positives or tests that fail when there is really no product problem are one of the biggest diseases of test automation suites. • A tester can spend endless hours reading failure logs, rerun the offending script and spend loads of time trying to analyze the failure. • How was the test failure caused ? (By timing issues, or did the script itself failed? Or something happen to the test environment ? • What you need is a tool which makes debugging of automated test runs easy; a tool which can help categorize failures and make test analysis easier. 4. Throw up False Positives!
- 11. Top 5 Pitfalls for Test Automation • You might land up with 2 teams of programmers; one writing code for the software product itself and the other writing code to test it. • The sign of a robust test automation tool is that it serves as an aid to a tester, to make the whole testing process more streamlined; however it can never prove to be replacement of the tester itself! • The ultimate aim of integrating a test automation tool into your test practice should be towards having a more seamless execution of test planning & execution strategy. 5. You can replace testers!
- 12. Thank You for your time….