Building a Test Automation Strategy for Success
10 likes8,503 views
The document outlines a comprehensive strategy for successful test automation, emphasizing the need for clear goals, readiness assessments, and organizational support. It identifies common reasons for test automation failures and maps out ideal attributes for organization, process, and technology. Recommendations include centralized automation efforts, proactive maintenance, and phased implementation to enhance effectiveness.
Building a Test Automation Strategy for Success
- 1. © Utopia Solutions Building a Strategy for Test Automation Success Lee Barnes, CTO Utopia Solutions
- 2. © Utopia Solutions Agenda •What is the end goal? •Why do we need a strategy? •What do we need to consider? •What should a strategy look like? Test Automation Strategy Overview Interactive Case Study
- 3. © Utopia Solutions SECTION 1 What is Our End Goal?
- 4. © Utopia Solutions4 Test Automation Success Criteria • Issue detection and recovery • Accurate verification • Unattended execution Reliable • Minimum sensitivity to application and test case changes • Test case definition separate from automation code Maintainable • Test coverage expanded efficiently • Automated test cases created by non- automation resources Scalable
- 5. © Utopia Solutions5 Test Automation Approach Test Automation Framework Automated Test Cases Test Results Test Definition Test Execution Non-technical / Test Analysts & Subject Matter Experts Technical / Automation Engineers
- 6. © Utopia Solutions SECTION 2 Why Do We Need a Strategy?
- 7. © Utopia Solutions8 Why Test Automation Fails Organizational • Lack of support • Unrealistic expectations • Disjointed org structure • Missing skill sets Process • Immature testing process • Segregated processes • Lack of test environment / data control Technical • Poor tool support for AUT • Under or over design automation approach
- 8. © Utopia Solutions9 Are We Ready?
- 9. © Utopia Solutions SECTION 3 What Do We Need to Know First?
- 10. © Utopia Solutions11 Test Automation Readiness Assessment Org • Structure • Support • Goals and Expectations • Resources and Skills Process • Test Maturity • Test Assets • Process Integration • Test Environment / Data Tech • AUT Technology • AUT / Tool Integration Assessment Topics
- 11. © Utopia Solutions Where Do We Get the Information? Assessment Input Stakeholders Existing Assets AUTs
- 12. © Utopia Solutions Ideal Organizational Attributes Structure • Centralized testing function • Testing is a peer of development • Automation roles are defined Goals and Expectations • Automation is an investment • 100% automation is not likely • Automation requires specialized skills • Automation will free up resources to work on high- value tasks Support • Executive management has visibility to automation • Automation has a dedicated budget Resources and Skills • Resources are dedicated to automation • Resources are skilled in automation as a process AND technology
- 13. © Utopia Solutions Ideal Process Attributes Test Process • Test objectives are clear and measurable • Test cases are defined and documented • Expected results are predictable SDLC Integration • AUT in a frequent release state with visibility to future roadmap • Visibility to system changes as early as possible in release cycle Test Environment • Test environment dedicated to automation • Test data controlled
- 14. © Utopia Solutions Ideal Technology Attributes AUT Technology • Multiple AUTs built on common technology and/or development framework • Consistent use of UI objects • Consistent navigation • Consistent presentation of user feedback (warnings, errors, etc.) Tool / AUT Integration • All UI objects supported by tool • Any alternative interfaces (web services, API, DB, etc.) supported by tool • All target end user environments supported by tool
- 15. © Utopia Solutions SECTION 4 Common Assessment Findings
- 16. © Utopia Solutions17 Organizational Distributed Testing / Automation Efforts App App App App • Uneven automation resource demand • Inconsistent approach • Duplicated efforts • Restricted skill growth and knowledge sharing Leads to…
- 17. © Utopia Solutions18 Organizational AUT A AUT B AUT C AUT D Centralized Test Automation Team Common Resources, Processes & Technology • Flatten demand curve • Accelerate skills development • Leverage reusable knowledge and technology Benefits Centralize Automation Activities
- 18. © Utopia Solutions19 Process Analyze Design Build Test Deploy No ability to identify impact of app changes on automated test suite Unexpected / Unpredictable Maintenance
- 19. © Utopia Solutions20 Process Analyze Design Build Test Deploy Capture potential impacts as early as possible to enable proactive maintenance Capture App Changes Analyze Impact Maintain Execute Automation Activities Application SDLC
- 20. © Utopia Solutions21 Environment Shared Test Environment Uncontrolled Test Data Integration Functional Performance Integration Test • Schedule conflicts • Unreliable execution • Unnecessary “hardening” of the test suite Leads to… Automation
- 21. © Utopia Solutions22 Environment Shared Test Environment Controlled Test Data Integration Functional Performance Integration Test Dedicated Environment for Automation Test Data Controls Dedicated Environment Controlled Test Data Automation • Restore to known state • Partition data related to specific test activities • Create required data each execution
- 22. © Utopia Solutions SECTION 5 What Does a Strategy Look Like?
- 23. © Utopia Solutions24 Mobile Test Strategy Topics Test Automation Strategy Process & Organization Environment TechnicalResources Scope & Roadmap
- 24. © Utopia Solutions25 Automation Strategy – Process and Organization • Political and financial support • Realistic expectations • Centralized automation function Key Organizational Components • Test objectives and coverage goals • Integration with SDLC → proactive maintenance Key Process Components
- 25. © Utopia Solutions26 Automation Strategy – Environment • Dedicated environment for execution • Controlled test data • Controlled code migration Key Environment Components
- 26. © Utopia Solutions27 Automation Strategy – Resources • Roles • Skills • Time Key People Components • AUT test environment(s) • Test suite development • Test suite execution Key Hardware Components • Test tool • Supporting utilities Key Software Components
- 27. © Utopia Solutions28 Automation Strategy – Scope • Test data setup • AUT / environment preparation • Complex / tedious results verification Test Support Tasks • Applications built on common technology • Applications with frequent releases & friendly UI’s Application Selection • Frequently tested • High test case to UI ratio • Predictable results • Easy to automate Test Case Selection
- 28. © Utopia Solutions29 Automation Strategy – Roadmap • Begin with required organizational and process changes • Overlap with base framework and minimal coverage • Enhance and add test coverage over small iterations Create Implementation Timeline Phase nPhase 1 Continuous Process & Organizational Improvements Base Framework Initial Coverage AUT 1 Framework Enhancements Expand Coverage Initial Coverage Phase 2 AUT 2 Expand Coverage Expand Coverage Initial Coverage Phase 3 AUT 3
- 29. © Utopia Solutions SECTION 6 Case Study
- 30. © Utopia Solutions31 Key Assessment Findings Organization • All enterprise testing activities currently performed centrally • Management believes automation is a tool implementation Process • Manual tests are well structured, but not maintained consistently • Rely on SME’s to execute tests Environment • Manual regression test performed in a isolated environment • Test data refreshed in all environments once per quarter Technology • All AUTs have browser based UI • Developed on disparate technologies • Test tool supports all AUT components Resources • Test organization consists of non- technical SME’s • System performance (mobile) • Device performance
- 31. © Utopia Solutions32 Key Summary Points • DO NOT automate without a clear vision and plan • Understand your readiness from these perspectives • Organization • Process & Environment • Technical • Resources • Implement in small phases
- 32. © Utopia Solutions33 Questions… …. and Answers! Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: lee.barnes@utopiasolutions.com Twitter: twitter.com/USI_LeeBarnes LinkedIn: linkedin.com/in/leebarnes Blog: www.RaiseYourMQ.com