End the Guesswork: How to Test the Right Devices, Platforms & User Conditions
Download as PPTX, PDF1 like214 views
The document discusses the complexities of continuous testing in digital environments, highlighting the need for a robust testing coverage plan that aligns with user conditions across platforms. It emphasizes the importance of utilizing relevant devices and operating systems to enhance testing efficiency and to stay ahead in the market, incorporating machine learning for better test automation outcomes. Additionally, it outlines the evolution and methodologies for testing, including a transition from traditional to machine learning-based testing approaches.
End the Guesswork: How to Test the Right Devices, Platforms & User Conditions
- 1. End the Guesswork: How to Test the Right Devices ,Platforms and User Conditions
- 2. HOUSEKEEPING • Please participate in our live poll • Ask us questions in the Q&A panel • We will send you the recording and slides 11/10/2017 2© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 3. AGENDA • Market Overview & Continuous Testing • How to develop a test coverage plan that reflects your actual users • Align cross-platform testing into the development cycle – what to test on at what stage • Plan your 2019 lab by testing on the relevant devices, operating systems, and browsers and keep ahead of the market! • Bonus: ML and Future of CT Eran Kinsbruner Dir., Lead SW Evangelist and Author, Perfecto Twitter: @ek121268 (https://twitter.com/ek121268) Blog: http://continuoustesting.blog LinkedIn: https://www.linkedin.com/in/eran-kinsbruner-4b47a81/
- 4. Continuous Testing for Digital Has Never Been More Complex 5/24/2018 4© 2018, Perfecto Mobile Ltd. All Rights Reserved. Augmented Reality Virtual Reality IOT Artificial Intelligence Mobile Progressive Web Continuous Testing for Digital
- 5. There Are Patterns for “Unstable” Test Automation 80% of issues have a pattern52% success rate 10% of devices, causing 80% of lab issues Lab 25% Orches tration 25% Scripts & FW 50% FAILURE REASON Objects Codding Time Other Scripts & FW issues Device in use No Device Orchestration issues Networking Stability Lock Other Lab issues What’s wrong With my Scripts What’s wrong With my Lab What’s wrong With my Executions
- 6. Mobile & Web Market Overview Coverage Methodology
- 9. What’s new? • Refreshed Country Data • Q4 2018 & Q1 2019 Mobile Calendars • Web, RWD Coverage Refresh • Market Overview Insights (Devices, PWA, Big Data) Perfecto’s Factors Magazine - 10th Edition Where to get it from: https://info.perfectomobile.com/v10Factors.html
- 10. Methodology
- 11. U.S & EU5 Recommended Coverage Q4 2018
- 12. Q42018 & Q12019 Calendars
- 13. Coverage Throughout the DevOps Pipeline (Process) 12/20/2018 13© 2015, Perfecto Mobile Ltd. All Rights Reserved. PLAN CODE BUILD TEST RUN New Functionality (Innovation) Users User experience
- 15. Key ML Use Cases In Test Automation Recognize objects Transcribe speech – Accessibility Make quality related decisions based on data Identify Trends and/or Patterns Security use cases – Identify signatures e.g. 12/20/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 16. Training AI the Testing Lingo 12/20/2018 16 © 2015, Perfecto Mobile Ltd. All Rights Reserved. PopupApp not installed Elementnotfound App didn’t launch Capability not supported App Crashed Keyboard Device in Use Trust
- 17. Training the Testing AI Engine – Image Recognition • Stabilize Execution By Eliminating Noise • Pop ups • Home screen • Keyboard open • Enhanced Reporting and Analysis 12/20/2018 17 © 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 18. Mindset and Workflow Changes in Test Automation 12/20/2018 18© 2015, Perfecto Mobile Ltd. All Rights Reserved. # Category 1 Test Authoring 2 Test Maintenance 3 Test Execution 4 Test Analysis 5 Tools Maturity 6 Available Integrations 7 Req. Test/Dev Skillset 8 Testing Env. 9 Testing Types 10 Supported App Types Test Engineers/Developers Manual Testers (Developers?) Traditional Test Automation Define Manual Flows, BDD Style, etc. Test Code & Reusable Functions (Java, JS, etc.) Define Objects, POM, use Object Spy Structured Page Based Test Steps Define Visual Validations and Assertions Longer Time To Develop, Complex Changes required pro-actively, SCM Supported Configure Env. (TestNG Data Provider), Execute Locally/CI/Cloud Flexible, OSS, Commercial High, Including Guidelines, Doc’s Plenty, OSS, Defect Management, etc. Medium-High IDE API, Load, Functional All Types ML Based Test Automation Record Test Flows (No Coding – in most cases) Groups Reusable Functions Objects Generated On-The-Fly (transparent to user) Flow Based Scenarios Part of Test Authoring Recording/Authoring Shorter Time, Advanced Capabilities Self-Healing/Correction Automatically handled/Object Scoring/Local SCM Controlled Configure Env., Execution Management Built-In, Locally/CI/Cloud Relies on ML/AI Tool Vendor in Most Cases Emerging Technology, Web More Advanced Than Mobile Most capabilities built-in, some exists Low-Medium ML UI Mostly Functional (E2E) & API Mostly Web
- 19. Q & A
- 20. Thank You