Test automation – the bitter truth
- 1. Test automation – the bitter truth Some unpleasant truths about “test automation”
- 2. Be a part of the show! Ask a question or leave your feedback at: https://www.sli.do/ using code #7346
- 3. Before we start, few prerequisites: Here’s a story …
- 4. What do testers imagine when they see “manual testing”? • Ineffective • Boring • Slow • Easy to perform by anyone • Mindless banging of keys
- 5. What do managers feel when they speak of “manual testing”? • “You are wasting too much time. “ • “We have too much testers” • “We can easily automate what you do.” • “You are not reliable”
- 6. How both managers and testers react when someone mentions “automation”? • Totally fangirling • We will reduce costs • We will have more time for “manual” testing • I will only hit the “Run” button and it will test on its own
- 7. What do testers imagine, when they speak of “test automation”?
- 8. About me • Senior Software tester at http://siteground.com • Lecturer at Pragmatic • Interested in software testing from scientific perspective • Would never say “Machines will replace us”, because I don’t believe it! • Blogger at: http://mrslavchev.com
- 9. Definitions:
- 10. Manual testing: Totally useless term Failure to provide value by executing “steps” Please, don’t dumb down your expertise by calling it “manual testing” Better term: Analytical testing, Human testing or just Testing
- 11. Automation testing: Made up “unicorn” term False belief that testing is simple sequence of steps Relies on false belief that testing is automate-able Better term: Tool assisted testing, automated checking, automation in testing, programmatic testing
- 12. No “manual” and “automation” testing • Testing is not a car transmission • Not all tasks in testing are equally automate-able • Testing is analytical human activity • Has nothing to do with hands or manuals
- 13. No hate intended • I actually love coding and resolving problems in a programmatic way! • I love testing, too • I hate it, when false expectations of testing are spread • We need to say some “bitter truths” in order to set things right • So think of them as “medicines”
- 14. Bitter truth #1: You can’t automate “all the tests”
- 15. Activities we perform during testing
- 16. Tasks that are easily automatable Confirmation / Verification Low level checking Precision Functional correctness Deterministic tests Solving machine problems
- 17. But automation sucks when dealing with: Exploration Unanticipated risks Qualitative evaluation Open ended questions Adaptive learning Solving human problems Inspired by:Things Could Get Worse- Michael Bolton
- 18. Bitter truth #2: Test automation won’t decrease cost of testing And that’s not even half the truth
- 19. Things we hear about automation: It will decrease time It will run faster It’s more reliable It’s cost effective Thing is … it’s only execution cost
- 20. How automation can increase cost: Cost of tools/time Development Maintenance Debugging your own code Make the madaf***r work!
- 21. Bitter truth #3: Don’t think automatic testing, think programmatic testing
- 22. Why does this distinction matter? Automated testing Implies testing is fully automatable Implies human testing is replaceable with tools Defines testing as simple execution of steps Looks at testing as test execution, only Programmatic testing Record and play tools = totally useless! Understanding of limitations Understanding of the approach Understanding of goals
- 23. Bitter truth #4: Your testing code sux, just like any other
- 24. Because…
- 25. Things that testers often forget: We are “mere mortals” like anyone else We design bad tests We sometimes have no ides what the f**k are we up to Testing code is not pixie dust
- 26. What should be our focus, instead? Why are you automating? Value? Development approach Infrastructure and maintenance Testing your own code
- 27. Bitter truth #5: Automation is not human testing on steroids
- 28. The nature of “automation”: We still use “test automation” without realizing what it means There’s plenty of analytical “brain” work automation can’t do. Humans won’t be replaced by it, unless they act as script executors.
- 29. Few important takeaways… It’s a trend and it won’t be easy People enjoy easy We can be successful by using tools for their right purpose “Manual” and “automated” testing are useless terms. Don’t think inside the labeled boxes
- 30. And most important… Try not to think like: Automate to replace human testing but rather: Automate to extend your testing and let it do whatever it can do better than you
- 31. Thank you very much! Any questions?
- 32. Feedback is love! Feedback is life! Share your feedback with us! https://www.sli.do/ Code #7346