Using real customer data to drive Testing
Download as PPTX, PDF0 likes89 views
This document discusses using real customer data and artificial intelligence to improve software testing. Traditional testing is time-consuming and misses defects because test cases are manually defined and do not adapt. A new approach called Digital Experience Optimization uses AI to automatically generate test cases based on real user journeys and behaviors. This continuous and adaptive testing approach finds more issues while requiring less effort over time compared to traditional structured testing.
Using real customer data to drive Testing
- 1. USING REAL CUSTOMER DATA TO DRIVE TESTING JUNE 2019
- 2. HOW A SOFTWARE BUG COSTS $45 MILLION AND A MARRIAGE Software is so heavily embedded in our day to day lives that a software failure can have severe implications Implications can be loss of reputation, revenue and in some cases lives
- 3. WHO IS TO BLAME FOR BUGS CREEPING THROUGH?
- 4. SHORTCOMINGS WITH TRADITIONAL, STRUCTURED TESTING • Test cases are manually defined ⇢ Time-consuming to produce ⇢ Includes assumptions & happy paths • Test cases have fixed structures ⇢ Same tests run every time ⇢ No use of results to improve coverage Shortcomings With Traditional, Structured Testing Structured testing Tests Identified defects Missed defects Like playing Battleship game without feedback: • Using the same grid locations for every game • Not using a hit to focus on sinking a ship • Only have a limited supply of pegs • Test cases are manually defined ⇢ Time-consuming to produce ⇢ Includes assumptions & happy paths • Test cases have fixed structures ⇢ Same tests run every time ⇢ No use of results to improve coverage Structured testing Tests Identified defects Missed defects Like playing Battleship game without feedback: • Using the same grid locations for every game • Not using a hit to focus on sinking a ship • Only have a limited supply of pegs Tests Identified defects Missed defects Structured testing Test cases are manually defined • Time-consuming to produce • Includes assumptions & happy paths Test cases have fixed structures • Same tests run every time • No use of results to improve coverage Like playing a Battleship game without feedback: - Using the same grid locations for every game. - Not using a hit to focus on sinking a ship. - Only have a limited supply of pegs.
- 5. THE COST OF TEST-CASE PRODUCTION Manual test case development Eggplant AI The Cost of Test-Case Production Linear growth of limited coverage with constant effort Time Test cases Test cases Time Effort Hundreds Millions Effort App1 App 2 W ebsite1 App 3 W ebsite2 System 1 System 2 Full coverage from only small initial effort Hugely reduced maintenance
- 6. A NEW APPROACH: DIGITAL EXPERIENCE OPTIMIZATION continuous quality Continuous Quality Operational Excellence DIGITAL EXPERIENCE TESTING DIGITAL EXPERIENCE MONITORING DEV OPS
- 7. AI EXPLORATION BASED ON REAL USER JOURNEYS Restaurants Location Search AI Exploration Based on Real User Journeys Homepage Hotels Flights Near Me Now Restaurants Near Me Now Hotels Restaurants Things to Do Rentals Hotels Location Dates Type Search Flights From Class To Search Passengers Back Back Back Back Rotate Screen Exit Exit AI model favors sequences of actions that are part of defined test cases.
- 9. USING REAL CUSTOMER DATA TO DRIVE TESTING JUNE 2019
Editor's Notes
- #7: Learn from real usage Identify and diagnose business–impacting problems