Agile Testing Days 2012 - Lean Mean BDD Automation Machine
- 1. Building a lean, mean, BDD automation machine Mike Scott & Tom Roden @MikeAScott @TomMRoden Agile Testing Days 20 November 2012 Potsdam, Germany SQS Group Limited
- 3. Safia– Archetypes and Templates, a voyage in automating acceptance A story about building an executable documentation system • Large hedge fund COTS Born out of implementation necessity • Crippling regression test burden • 6-8 month cycle daily release Results: • 80 times defect reduction • and again Then did it • Same patterns automation again… platform 3
- 4. Real data from the field The problem quantified Time taken for Release Regression testing 120 700 100 601 600 100 545 500 80 479 78 Man Days Effort 67.5 393 400 # Test Suites 371 60 311 300 284 40 40 217 200 151 106 20 23 15 100 64 12 28 25 5 0 0 0 3 0 0 February April August November February April May June July August Month Man Days Effort for Regression Test # Test Suites # Automated Suites 4
- 5. Where it came from The confidence problem! First Approach to Acceptance Testing: UAT in a phase, after a period of development 5
- 6. 6
- 7. What it felt like… 7
- 8. Building Integrity in Becoming acceptance driven… Ready User Backlog Specify tests Story / examples Automate acceptance Develop Demo / tests Showcase Session Customer Based Verification Testing Stakeholder(s) & BA Developers Testers 8
- 9. Real data from the field The initial effect of automated acceptance testing Time taken for Release Regression testing 120 700 100 601 600 100 545 500 80 479 78 Man Days Effort 67.5 393 400 # Test Suites 371 60 311 300 284 40 40 217 200 151 106 20 23 15 100 64 12 28 25 5 0 0 0 3 0 0 February April August November February April May June July August Month Man Days Effort for Regression Test # Test Suites # Automated Suites 9
- 10. What we gained from this Knowledge and experience to design a framework A set of design patterns • binding tests to investment apps Inject knowledge • from business specialists Vanilla acceptance tests • easy to customise Living documentation • tests as specifications 10
- 11. How Safia Works Illogical Architecture Diagram Fitnesse fixtures Maps SAFIA terminology to system terminology Safia fixtures e.g. Calypso, Beauchamps, Summit Maps investment house House System terminology to SAFIA Adaptor Adaptor terminology e.g. Trade Capture = Trade Entry API Trading system SAFIA automates at the server level via the API 11
- 12. How Safia Works Creating automated test cases using Safia e.g. Run a P&L report e.g. Update a legal entity FX, Bond, Equities Amend SDI IRS, FRA, CDS Test Archetype Instrument Amend fees “Test Function” Template Create a trade IRS template Apply Template & Archetype Test Create a valid IRS trade 12
- 13. Demo Let’s put it to the test 13
- 14. Why use a framework like this It brings benefits on different levels Increased collaboration Reduced Cost of Ownership Accelerating implementation & learning Benefits Teaching & coaching aid Anyone can interact with it 14
- 15. In retrospect… It was easy, right? 15
- 16. Real data from the field The path to faster release Time taken for Release Regression testing 120 700 100 601 600 100 545 500 80 479 78 Man Days Effort 67.5 393 400 # Test Suites 371 60 311 300 284 40 40 217 200 151 23 20 106 15 100 64 12 28 25 5 0 3 0 0 0 0 February April August November February April May June July August Month Man Days Effort for Regression Test # Test Suites # Automated Suites 16
- 17. How we wrote the tests Tests as specs, the principles of test design Self-documenting Based on testable statements About business intentions What makes a good acceptance test? Anyone can understand it Concise and A specification granular not a script 17
- 18. Reducing the Cost of Ownership ~One third is system understanding 18
- 19. Points to consider when automating Theory ‘aint practice Start early & iterate frequently Distil and simplify Tests or Documentation? DON’T PANIC !!! 19
- 20. “Brace yourself Daphne…” 20
- 21. Thank you for listening Questions Please evaluate our presentation by using the evaluation booklets which you can find in your conference bag. Thank you! 21
- 22. Mike Scott & Tom Roden mike.scott@sqs.com tom.roden@sqs.com SQS Group Limited 7-11 Moorgate, London EC2R 6AF Internet: www.sqs.com SQS Group Limited