Best Practices for Accelerating Continuous Testing
- 1. Seven Best Practices for Accelerating Continuous Testing Marc Hornbeek Senior Solutions Architect
- 2. 2Spirent Communications “There is no continuous delivery without testing” Dev Ops
- 3. 3Spirent Communications Fast CT is required to race through the many test steps in the pipeline from Pre-Flight through to Delivery
- 4. 4Spirent Communications CT infrastructure must be stable and accurate because test failures at any point in the pipeline cause delays or roll- backs
- 5. 5Spirent Communications Consequences of poor CT practices 80% failure rate when using current tools IDC Fortune 1000 survey, December 2014
- 6. 6Spirent Communications Accelerated Continuous Testing Best Practices Principles Comprehensive & Results Test fast & Don’t Stop Powerful Relevant
- 7. 7Spirent Communications CT best practices are across the pipeline “Shift Left”
- 9. 9Spirent Communications Focus on customer relevant defects not quantity of defects “Quality is everyone’s Responsibility.” Dr. W. Edwards Deming
- 10. 10Spirent Communications Pre-Flight Team-workDesign for CT CT End-to-End Culture
- 11. 11Spirent Communications CT Process design Fault tolerant Obfuscation Intelligent dashboards CT-Ready Infrastructure
- 12. 12Spirent Communications CT-Ready Tools Restful APIs Cache and pipeline Virtual and physical Program agnostic Large scale
- 13. 13Spirent Communications CT Tools Integration Pre-Flight CI Tests Integration Regression System Deployment
- 14. 14Spirent Communications Orchestrate Test Topologies Physical and Virtual Utilization measurement Topologies Test Automation
- 15. 15Spirent Communications Accelerate Test Execution Powerful servers Test design Pipeline Scale horizontally Thresholds Dynamic test schedules
- 16. 16Spirent Communications Fast Relevant CT Analytics Continuous Monitoring (CM) Relevance Dashboards, Alerts, logs, thresholds Aggregation Snapshots and trends Telescope for diagnostics
- 17. 17Spirent Communications Case Study 360 developers, six sites 36 MLOC+, 100+ features 86 build targets complex interdependencies 200+ topologies 12K test cases 80+ deployment packages (physical and virtual) 1,280,000,000 test possibilities
- 18. 18Spirent Communications The primary contributing factor was CT and CM ! CT Case Results Metric Major release (#months) Minor release (#weeks) # Features Defects Integrations / day Tests / day Automated tests After 3 2 150 10 100 10 85% Before 6 4 113 1260 0.5 0.3 5%
- 19. 19Spirent Communications Best practices toolBest Practices Assessment Tool simplifies the assessment process.
- 20. 20Spirent Communications Summary 7 CT best practices emphasize speed, don’t stop, comprehensive and relevant. Benchmark against Best Practices Blueprint and DevOps Assessment. Don't fall into the “do-CT- yourself / NIH” trap! “There is no continuous delivery without testing”
- 21. 21Spirent Communications www.spirent.com/clear 1) CT Blueprint white papers 2) CT eBook 3) CT Assessment Tool Take-aways http://www.spirent.com/go/AMR
- 22. 22Spirent Communications How can Spirent help?: Test tools CT Orchestration Lab Management Professional Services
- 23. 23Spirent Communications © Spirent Communications, Inc. All of the company names and/or brand names and/or product names and/or logos referred to in this document, in particular the name “Spirent” and its logo device, are either registered trademarks or trademarks pending registration in accordance with relevant national laws. All rights reserved. spirent.com Thank you Seven Best Practices for Accelerating Continuous Testing Marc Hornbeek Senior Solutions Architect Marc.Hornbeek@spirent.com Twitter: mhexcalibur