Why Your Next QA Job Might Be in Ops
- 1. Why Your Next QA Job Might Will Be In Ops Edward Rousseau @elrousseau
- 2. Who Am I? • 20+ year tech vet • QA, Product Owner, Manager, DevOps Engineer • QuintilesIMS, CureForward, SimpliSafe, Red Hat, Rockwell Automation, Startups • Passionate about efficient engineering process and kanban, philosopher, occasional DJ, lover of whiskey • t: @elrousseau • BDO slack: @erousseau 2
- 3. 3
- 4. A follow on to ^^^
- 5. What are we doing here? • Today’s CI/CD cloud native world has killed traditional QA / Testing in organizations • Infrastructure code needs testing like product code • A QA mindset fits in modern “Ops” responsibilities 5
- 6. Who Testers Are Was It Good for You? Linda Wilkinson 6
- 7. What is a Tester? • Has a macabre sense of humor • Likes breaking things • Curious • Experimental • Quick learning • Apolitical • Questions everything 7
- 8. – Me “Software sucks. We make it suck less.” 8
- 9. Not QA’s Job • Quality Police • Release Pope • Gatekeeper • Make sure the release ships 9
- 10. QA’s Job • “QA as mirror” - Jeff Sussna QA = Quality Advocate • has holistic view of the system and its parts • “To inform the business the state of the product in order for it to make intelligent business decisions” - Me • Design and run tests and experiments, analyze and interpret results, provide information • Fight confirmation bias 10
- 11. QA and Ops • QA - correctness, defect escape • Ops - stability, availability • Production breakage impacts both • Both mistrust Dev claims (professional paranoids) • Both undervalued by biz until something goes wrong 11
- 12. Traditional QA • Code thrown over the wall • QA considered less than Dev • Manual Testing (spreadsheet jockeys) • Test Automation not widespread • Proxy for the customer inside engineering #itwasthe90s
- 13. MTBF 13
- 14. Why? • COF (Cost of Failure) high • Rework expensive • Product physical • QA breaks things so customer doesn’t have to 14
- 15. Agile Revolution • Agile practices and online distribution spreads • Widespread test automation • QA writes tests on Scrum team • Or QA is outsourced • Customer engaged more directly in engineering process
- 16. • Operational capability is increasingly important to the business • Platform is the product • Ability to deploy production changes cleanly has business impact (smaller maintenance windows) • Ops work more customer visible (Prod Ops) • Perceived need for QA wanes 16
- 17. Disruption • Speed trumps correctness • MVP over comprehensive release • Build-Measure-Learn • Pivot 17
- 18. MTBF
- 19. MTTR
- 20. Why? • COF low • Rework fast and cheap • No physical product • Feedback from customer baked into the process 20
- 21. Automation Fetish • Test automation viewed as the only valuable QA activity • So why not just let the developers do it • Testers get trapped in tools - lose system view 21
- 22. Rise of the Cloud • Cheap elastic compute levels the playing field • Established players disrupted • Time to Market more important than ever
- 23. Why? • COF is lowest in history • Rework faster and cheaper than ever • Elastic resources provide maximum flexibility • Deployment automation increases speed (infrastructure as code) 23
- 24. Automation Apocalypse • Machine learning and computer vision are good enough for visual test tasks today • Automatic test generation is quickly improving • The time of humans doing rote verification is nigh 24
- 25. • Operational expertise is critical to the business • Pipeline is the product • Reliable repeatable delivery is a core business capability (downtime an anomaly) • Software system integration skills required • QA mindset is ??? (do we even need it?) (spoiler alert: yes) 25
- 27. What Next? • QA mindset still valuable - may not be called QA or test • Infrastructure is code. It needs to be tested better than product code • Performance and Security more important • Complex Distributed System behavior needs to be confirmed 27
- 28. Infrastructure as Code • Code needs to be tested wherever it is • Resilience of pipeline enables speed of the business - you need to trust it • Software supply chain is important. 3rd party libs and services need to be verified 28
- 29. Performance • Performance testing important for customer experience for web, IoT and mobile • Not just raw speed but right- sized infrastructure and scalability • Inefficiencies cost real money (low conversion rates, over- instanced infrastructure) 29
- 30. Security • Do we need to test for security? EQUIFAX • Pen testing is testing
- 31. Compliance • Compliance regimes have strict privacy and audit requirements. Testing affirms compliance • Violations have serious business consequences 31
- 32. User Intelligence • If it can be done users will do it • Techniques exist to capture real user data from production (A/B testing, RUM) • Need to be properly designed and analyzed to avoid misinterpreting data 32
- 33. Chaos Engineering Monkeys for the Monkey God
- 34. Chaos Engineering • Complex Distributed systems cannot be fully replicated in a lower environment • Must be able to test in production to get reliable information • Destructive experiments prove the resiliency of the system
- 36. IoT • Hardware means MBTF is important again • Update / replacement has costs and implications (shipping, warehousing, security, physical replacement) • CoF is higher 36
- 37. Conclusion • Increasing amounts of code in infrastructure and other “ops” related domains needs to be tested • A QA mindset is helpful for these tasks • COF will determine the level and rigor of that testing 37
- 38. • twitter: @elrousseau • BDO slack: @erousseau • email: edward.rousseau@gmail.com • cocktail bars in the greater Boston Area and worldwide