![12
23%
Tests
Shi]ed
to
Business
Layer
REDUCED
COST
&
EFFORT
WITH
IMPROVED
QUALITY
BENEFITS
UI
-‐
8,500
BL
-‐
2,500
DISTRIBUTED
TESTS
15%
ReducIon
in
Monthly
Person
Efforts
35%
ReducIon
in
License
Costs
$500K
ANNUAL
SAVINGS
$200K
ANNUAL
SAVINGS](https://image.slidesharecdn.com/t1510-170606153020/85/Applying-Agile-Principles-to-Test-Automation-Development-14-320.jpg)
Applying Agile Principles to Test Automation Development
- 1. T15 Test Automation 5/11/17 13:30 Be More Effective: Test Automation below the UI Presented by: Ashish Mehta Infosys Limited Sohail Farooqui Allstate Insurance Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ info@techwell.com -‐ http://www.starwest.techwell.com/
- 2. Ashish Mehta Ashish Mehta has more than fifteen years of global experience in software quality assurance and development. He has worked with clients to establish independent QA organizations, multi-‐stack test competencies, and mature testing practices in system test automation, middleware automation, data-‐warehouse, and end-‐to-‐end integrations. Ashish's passion for new technology has helped him collaborate with architects and developers in designing and developing optimal test solutions. Sohail Farooqui As an Enterprise Test and Release Manager in the Allstate Insurance strategy group, Sohail Farooqui provides strategic direction and thought leadership in the functional, regression, performance, and service architecture testing spaces across several mission critical and supporting applications while also serving on the core leadership team providing guidance in the conversion of Allstate testing organization from a waterfall to an agile methodology. As a Strategist, Sohail is critically focused on quality and balancing cost to business with testing efficiency.
- 3. BE MORE EFFECTIVE MAY 7–12, 2017 | ORLANDO, FL BELOW THE UI Sohail Farooqui & Ashish Mehta TEST AUTOMATION
- 4. 2 High Test Automa;on Scripts Maintenance Efforts High Test Cycle Times Increased Tool Licenses Cost Large Enterprise Insurance Policy Administra;on System demanding Con;nual Regression to Maintain Op;mal Product Quality THE PROBLEM
- 5. 3 11,000+ REGRESSION SCENARIOS 10% Year-‐on-‐Year increase in Scripts Volume 3,000+ HOURS PER MONTH Higher Test Cycle Times Increasing Licenses Cost UI based regression automa;on scripts naviga;ng across several pages to complete policy transac;ons CURRENT STATE -‐ UI REGRESSION TESTING $500K ANNUAL LICENSES COST
- 6. PROBLEM DISCOVERY No visibility in the tests UI based Test Automa;on Applica;on UI Business Layer Components Peripheral Applica;ons No visibility in the tests Higher defects concentra;on UI Layer Business Layer 78% 22% Defects Distribu;on
- 7. PROBLEM SOLVING NEW AUTOMATE AT BUSINESS LAYER (BELOW THE UI) RENEW OPTIMIZE EXISTING UI REGRESSION SUITE
- 8. AUTOMATE AT BUSINESS LAYER BELOW THE UI Applica;on UI Eliminate UI dependency Business Layer Components 1 Peripheral Applica;ons Test run 2 Test run 1 2 3 Planned Release (Test version) Baseline Code (Prod version) Reset transacIon (s) dataset to iniIal set 1 Run dataset through baseline code 2 Run dataset through test code 3 Auto compare results
- 9. AUTOMATE AT BUSINESS LAYER -‐ BELOW THE UI MIMIC UI TRANSACTIONS AT BUSINESS LAYER Goal Guiding Principles Exploit the system capability Intercept at business layer Supply historical system of records
- 10. Not all UI tests can be moved to business layer, since UI layer is also a source of failure point. Regression tests at UI and business layer need to co-‐exist. SHIFT THE TESTS FROM UI > BUSINESS LAYER OPTIMIZE EXISTING UI REGRESSION SUITE Categorize UI workflows UI tests covers unique UI elements Remaining tests moved to business layer Iden;fy unique UI elements
- 11. 9 CO-‐EXISTENCE UI & BUSINESS LAYER AUTOMATION Test distribu;on between UI & business layers 11,000 Test Scenarios 2,500 at Business Layer 8,500 at UI Layer NEW RENEW
- 12. 10 EARLY VALIDATION TIME/TRANSACTION VALID DEFECTS # OUTCOMES BUSINESS LAYER AUTOMATION 2 weeks & 55% defects 30 Min 17 Min 35% 65% COST, QUALITY & TIME COMPARISON UI BASED AUTOMATION
- 13. 33% 14% 52% BL DEFECTS UI DEFECTS COMMON DEFECTS BETWEEN BL & UI BUSINESS LAYER AUTOMATION UI BASED AUTOMATION OUTCOMES TEST EFFECTIVENESS
- 14. 12 23% Tests Shi]ed to Business Layer REDUCED COST & EFFORT WITH IMPROVED QUALITY BENEFITS UI -‐ 8,500 BL -‐ 2,500 DISTRIBUTED TESTS 15% ReducIon in Monthly Person Efforts 35% ReducIon in License Costs $500K ANNUAL SAVINGS $200K ANNUAL SAVINGS
- 15. 13 TECHNOLOGY MIGRATION & NEW CONSUMER ONBOARDING LEVERAGE FOR END-‐TO-‐ END/REGRESSION TESTS FUNCTIONAL TESTING SHIFT-‐LEFT: EARLY DEFECT DETECTION DURING FUNCTIONAL PHASES PERFORMANCE TESTING TEST PERFORMANCE AT BUSINESS LAYER IN ISOLATION FOR BETTER BOTTLENECK ANALYSIS SOLUTION PROSPECTS
- 16. Q&A
- 17. sfarp@allstate.com Please contact us at amehw@allstate.com