Quality in dev ops east 2017
Download as PPTX, PDF1 like172 views
The document outlines 5 stages of digital quality maturity for organizations: 1) Automate testing, reuse test assets, and scale testing across devices and browsers. 2) Shift testing left into development to provide faster feedback and prioritize important tests. 3) Prioritize test executions based on risk and business impact. 4) Provide continuous testing visibility across development, QA, and production for real-time monitoring. 5) Enable remote debugging and support directly from the cloud.
Quality in dev ops east 2017
- 1. 5 Stages of Digital Quality Maturity Amir Rozenberg, Director of Product
- 2. BUT: I still have ~20 GitHub projects! • Product manager by day, coder at night • Responsible for the core platform at Perfecto • Publications: DZone, CIO Magazine, MobileTestingBlog.com etc. • I toy with • Security & performance testing • Globalization • Elasticity • Music and video streaming quality • … Amir_rozenberg
- 3. 11/8/2017 3© 2015, Perfecto Mobile Ltd. All Rights Reserved. Agenda Digital Trends in FinServ Digital Quality: Current, Desired State Implementing Quality into devOPS (CI/CD) Q&A
- 6. User-Centric Drivers Image Source: ReMarkgroup.com Multi Channel Multi Screen
- 7. User-Centric Drivers Multi Channel Multi Screen Personalization Promotions
- 8. User-Centric Drivers Multi Channel Multi Screen Personalization Promotions Streamlined Enriched
- 9. 50% of CEOs expect their industries to be substantially or unrecognizably transformed by digital. Source: Gartner, 2016
- 10. Current State of Digital Quality Outdated Quality Practices • Mostly repetitive manual testing • Post Sprint • No/outdated, local lab • Outdated bank of scripts and framework • Partial reporting & lack of direction Team Impact • Risk-based approach • Expensive, latent feedback • Poor app performance • "Over the wall" attitude • Frustration and fatigue Business Impact • User frustration, adoption, usage and churn • Impact on top-level metrics: share value, competitive leadership, brand etc. • Sense of the need for a revolution
- 11. I’m dreaming of …continuous testing ;)
- 12. Quality @ devOPS Principles • Skills elevation • Dev-QA-OPS: one team • Dev agile process • Everyone should work at the same rhythm • Real time decisions • Real time bug fixing • Developers test • Automation AI: • Green is Green • 3 strikes out • Smart retry • Always on, scalable & stable lab • Enabling framework • Big data reporting • Transparency & alignment
- 13. Quality @ DevOps is an Agile Process 11/8/2017 © 2015, Perfecto Mobile Ltd. All Rights Reserved. Coverage DailySprint Smoke Full Coverage before Robustness Robustness Before Coverage Most of the market • Sprint level execution • Half-way to coverage Execution Frequency
- 14. Current State Local Devices/Lab Manual Testing (CoE) Limited/Local Appium/Selenium APM/ Production Monitoring Waterfall/Months
- 15. Stage 1: Automate, Reuse, Scale Local Devices/Lab APM/ Production Monitoring Use your Automation Framework Execute tests every build Connect the cloud to your IDE Android Studio • IntelliJ • Eclipse • Visual Studio
- 16. Stage 1: Automate, Reuse, Scale • Functional Testing • Increase automation: OSS+, Framework, Reuse (one lab) • Increase coverage: Wind Tunnel, Audio/Video, Localization, IVR, sensors • Parallel execution • Non-Functional Testing • Single user performance • Accessibility • Security = Multi-screen End user conditions
- 17. Stage 2: Shift left & Prioritize Local Devices/Lab APM/ Production Monitoring Use your Automation Framework Execute tests every build Connect the cloud to your IDE Android Studio • IntelliJ • Eclipse • Visual Studio
- 18. Stage 2.1: Shift left, carefully Stability, visibility & trust => Value CoE Dev
- 20. Chatbot Example: BOFA’s Erica Audio Record Perfecto CloudYour script Speech to Text Text out Audio out 1. Show my account balance Validate (BOFA Erica is saying): 1. You have $300 in your account WithReference Audio Quality Audio Quality Score, log 1. Show my account balance 2. Transfer $10 to Dan Visual Validation Audio InjectText to Speech Text in Validate (BOFA Erica is saying): 1. You have $300 in your account 2. Please confirm transfer $10 to Dan Audio in
- 21. Prioritizing Chatbot Testing • Input: pre-recorded samples (accents, stutter etc.), audio inject • Validation: Audio record, Speech to text, audio quality • Input: Speech to text, audio inject, languages and genders • Validation: Audio record, Speech to text • Input: Speech to text, audio inject • Validation: Native/Visual text • Input: Native Objects • Validation: Visual (UX timer) • Input: Native Objects • Validation: Same Functional Testing Responsiveness Basic Voice Testing 80% Voice 100% Functional AI Testing User Experience Impact Automatable with Perfecto
- 22. Stage 4: Continuous Quality Visibility Local Devices/Lab Use your Automation Framework Execute tests every build Connect the cloud to your IDE Android Studio • IntelliJ • Eclipse • Visual Studio Quality Visibility One report, all platforms, see the trend ‘Test in Production’ Know it • Triage it • Fix it
- 23. Centralize Quality: Visibility, Traceability, Fast Analysis • One cloud BI Reporting system from executive to root cause analysis for engineers • Full Flexibility to view any granularity • One click drill down with full traceability between reporting levels • Out-of-the-box digital relevant filters + Build any custom filters • Collaboration capabilities for remote joint analysis • Digital View • Fast analysis • Fast feedback loop
- 24. CI Dashboard
- 25. Centralize Quality: Include Production “Shift Right”
- 26. Stage 5: Debug & Support Use your Automation Framework Execute tests every build Connect the cloud to your IDE Android Studio • IntelliJ • Eclipse • Visual Studio Quality Visibility One report, all platforms, see the trend ‘Test in Production’ Know it • Triage it • Fix it Remote Debug to the Cloud ADB • Chrome inspect
- 27. Pragmatic plan to achieve Digital Quality Automate everything Prioritize Executions Report Early Streamline processes Perfecto!
- 28. Coming soon: Smarter Quality in DevOPS • Prioritizing test executions (in a shrinking timeframe) • AI in test orchestration • “Impactful Code”: Closing the loop from planning to production on code areas that mandate more attention • Senior developers • More testing • Attention in production
- 30. @Amir_Rozenberg