Winning strategies in Test Automation
0 likes607 views
The document discusses strategies for successful test automation, outlining definitions, tools, and best practices in the field. It highlights the importance of selecting the right tools, avoiding common mistakes, and ensuring proper maintenance and skills within automation teams. The content also emphasizes the need for human oversight in testing processes to complement automated methods.
Winning strategies in Test Automation
- 1. XBOSo&, Inc. All Rights Reserved. 1 Winning Strategies In Test Automation
- 2. XBOSo& Founded in 2006 Dedicated to software quality Software QA Consulting Software Testing Offices in San Francisco, Beijing and Amsterdam XBOSo&, Inc. All Rights Reserved. 2
- 3. House Rules Everyone except the speakers are muted QuesDons via the gotowebinar control on the right side of your screen or through TwiIer @XBOSo& QuesDons can be asked throughout the webinar -‐ we’ll try to answer them at the end. You will receive info on recording a&er the webinar XBOSo&, Inc. All Rights Reserved. 3
- 4. Meet Our Speakers • CEO and Founder of XBOSoft • Relevant specialties and passions – Software quality process, evaluation, measurement and improvement – Software quality in use / UX design – Mobile User Experience and usability – Cycling and travel XBOSo&, Inc. All Rights Reserved. 4 Philip Lew • Guest Speaker • 15+ years in Testing and QA management • International conferences speaker • STP Community Advisory Board Member • BBST Instructor • Passions: • Cross-functional team collaboration • Testers’ skills development • Quality advocacy Anna Royzman Sabrina Gasson • Marketing Manager of XBOSoft • Emails you all regularly to join our industry hot topic webinars • And invites you all to download our latest trends in software testing whitepapers.
- 5. MY EXPERTISE • Manager of so&ware testers and automaDon engineers over variety of projects, in both agile and waterfall • Leading AT SIG (special interest group) in Technology department of my organizaDon • 15+ years experience with home-‐grown test automaDon frameworks, commercial and open source tools XBOSo&, Inc. All Rights Reserved. 5
- 6. POLL QUESTION WHAT’S YOUR PRIMARY RESPONSIBILITY: • A. MANAGER RESPONSIBLE FOR TESTING • B. AUTOMATION ENGINEER • C. TESTER • D. NONE OF THE ABOVE XBOSo&, Inc. All Rights Reserved. 6
- 7. DEFINITION OF AUTOMATION UTILIZATION OF TOOLS, FRAMEWORKS AND SCRIPTS THAT ASSIST WITH TESTING OR CHECKING TASKS XBOSo&, Inc. All Rights Reserved. 7
- 8. EXAMPLES • Unit tesDng frameworks • Test controllers • Recording and playback • Performance tesDng tools • Security tesDng tools • AnalyDcs collectors • Screen capturing tools • Code coverage tools • Protocol tesDng tools • Monitoring tools • Database tesDng tools • Simulators, log helpers, debuggers • Data populaDon scripts: setups, etc. • Keystroke recording tools XBOSo&, Inc. All Rights Reserved. 8
- 9. AUTOMATION IS A HELPER – CreaDng a ‘safety net’ of business-‐criDcal workflows and logic that must work every Dme – Allowing to check code on various levels: from unit components to distributed systems – AssisDng humans with humongous or repeatable tasks (stress or load tesDng, system integraDon tesDng, data tesDng, 3rd party cerDficaDon etc.) – Debugging, analyzing or replicaDng failures – Monitoring running programs and capturing results for later analysis XBOSo&, Inc. All Rights Reserved. 9
- 10. AUTOMATION IS A CODE • It’s created by humans: IT HAS BUGS • It needs development and debugging Dme • New/changed funcDonality affects test suite, it requires maintenance to keep up to date • It needs skills • It has upfront costs • There is always version 2 XBOSo&, Inc. All Rights Reserved. 10
- 11. SKILLS AND RESPONSIBILITIES • Toolsmith (A tester? possibly!) • AutomaDon Engineer (A tester? Not necessarily!) • Scripter (A tester? Not necessary, but it’s good to be one) • Customer of automaDon (Must be a tesDng expert and has skills in tesDng strategies) XBOSo&, Inc. All Rights Reserved. 11
- 12. TEST AUTOMATION STRATEGY • CriDcal coverage • Low maintenance: has to be ready for next iteraDon/ release with low input • Easy to run on demand (one buIon click, ways to select tests for specific scenarios, pre-‐selected suites for different coverage) • Fast (mulDple environments, distributed, overnight, etc.) • TargeDng for ‘0 tolerance’ – failures are acceptable for bugs in the product only XBOSo&, Inc. All Rights Reserved. 12
- 13. QUICK POLL WHAT AUTOMATION TOOLS/FRAMEWORKS DO YOU USE: A. LICENCED TOOLS B. OPEN SOURCE C. HOME-‐GROWN D. MIX OF THE ABOVE E. NONE XBOSo&, Inc. All Rights Reserved. 13
- 14. SELECTING TOOLS AND METHODS • What’s your long term and short term goals? • Build, buy or open source? • What to look for in a tool • Who will be supporDng it: – Do you have specialists? – Is that their sole responsibility? – What will happen when your company has RIF or specialist leaves? XBOSo&, Inc. All Rights Reserved. 14
- 15. COMMON MISTAKES TO AVOID • AutomaDng new versions through GUI (it can work well with legacy version for tesDng upgrades) • Hardcoding values • Trying to test all aIributes all the Dme. The test failing on insignificant aIribute is noise. All tests failing due to one aIribute is more noise and makes AT unusable. • Product code replicas • Get carried away with cool features of less value • StarDng automaDon too early or too late • Using only one tool for everything • Assuming automaDon replaces human interacDons XBOSo&, Inc. All Rights Reserved. 15
- 16. MY ADVICE BASED ON MANY YEARS EXPERIENCE: XBOSo&, Inc. All Rights Reserved. 16 • AT is good when is up to date; it should be available when team needs it • Cross-‐team integraDon: • build simulators for majority of tests • limit integraDon tests to the most criDcal paths. • Building a framework for your own product: • Try to test each layer separately instead of tesDng everything from one layer • Try to avoid heavy UI automaDon as much as possible. • Take responsibility! As a tester, demand that the features important to you are prioriDzed by automaDon engineers
- 17. QUICK POLL ARE YOU WINNING WITH YOUR TEST AUTOMATION: A. YES B. NO C. WE DON’T HAVE IT XBOSo&, Inc. All Rights Reserved. 17
- 18. RISK MITIGATION HINTS • If you don’t have a toolsmith in your team, consider lending from other teams or asking programmers • Don’t replace exploratory tesDng with AT (it needs human brains) • Don’t replace usability tesDng with AT (it needs human emoDons) • Always have “plan B” in an emergency such as: • test server failure • DB permission errors • blocking issue takes too long to fix right before release • no electricity XBOSo&, Inc. All Rights Reserved. 18
- 19. Post your questions on Twitter and we'll answer them @XBOSoft Join us to keep updated on all our webinars, reports and whitepapers: facebook.com/xbosoft +xbosoft linkedin.com/company/xbosoft We post regularly on our blog – check us out! http://xbosoft.com/software-quality-blog/ Why not download our free Whitepapers, available here: http://xbosoft.com/knowledge-center/ You will receive an email with information on slides and recording. Any further queries regarding our services or ideas for future webinars please email us! Services@xbosoft.com Q+A www.xbosoft.com