The shift left strategy
- 1. THE “SHIFT LEFT” STRATEGY OVERCOMING THE DOWNWARD SPIRAL OF DOOM Darian Rashid darian@clearskytestautomation.com
- 2. 2
- 3. 3 Late? Too many defects? Not enough content delivered? How did we get here?
- 4. 4 DeployDesign Development TestRequirement Create Test Cases LETS REWIND First time we validate requirements Development has most likely overrun its time Test capacity is constrained Not enough people and time to run System, Integration and Regression effectively
- 5. 5
- 6. 6 Fix easy issues first “Cannot replicate” Test “Coverage” Technical debt mountain How many issues can be differed? How many issues from previous releases continue to be ignored?
- 7. AMBIGUITY IN REQUIREMENTS? 7 Leads to missed expectations and rework Discovered well into development cycle Each reworked item could: Impact delivery schedule Incur fines WELL, YES AND NO… Best case: Spend time arguing that the delivery met the requirements Worst Case: Implement CRs during testing • Ambiguous requirements • Test cases created after development has begun
- 8. “We just finished up our sprints. Now we’re gonna test.” BUT..
- 10. 10 THE CODEBASE Brittle code Deeply coupled, interconnected web Usually new features on top of legacy code Refactored last in the Regan era
- 11. 11 THE TEAM
- 12. 12
- 13. 13
- 14. 14 EMERGENCY PATCHES Haven’t accounted for these in the release Are already overcommitted for the next release Cannot drop deliverables RESULT: A compressed time frame for the next release
- 15. 15 THE “DOWNWARD SPIRAL OF DOOM” Ambiguous requirements & over- commitment & technical debt Compressed development timeframe Compressed Q/A timeframe Rushed delivery & slashed test matrix Rework & brittle code Low quality release Production issues & patches EACH RELEASE WORSE THAN THE LAST
- 16. 16
- 17. Prove Delivery Run Manual Tests Create Test Specifications Create Development Design Documents Develop Code/ Execute Tests, Daily Create Customer Requirements THE PARADIGM SHIFT PROVE SOFTWARE DELIVERY BY: 1. Creating test specifications BEFORE development begins 2. Execute ALL test cases DAILY, even during development 3. Measure test coverage against requirements DAILY Customer Requirements Objectively measurable test specifications CONTAIN ISSUES
- 18. CONVENTIONAL GHERKIN Create Adult Customer Profile with Preferred Phone as Home Phone 1.Launch the eCustomer Application 2.Click on ‘or register’ link GIVEN I am at the Home page WHEN I click on the “Register Now” Link THEN I will be taken to the Register screen AND I will see the following: 3.Enter the following Mandatory Details: (a) enter the valid 10 digit Home phone number (b)Preferred Phone: Home (c)Preferred Contact Method: Email (d)Enter the other details given in the default demographics section of test case initialization table WHEN I enter the following: THEN I will be taken to the Contact Information Screen AND I should see the following Title First Name Middle Name Last Name … Answer (blank) (blank) (blank) (blank) … (blank) First Name Middle Name Last Name … Answer Miss Bethany Jones … 9021 First Name Middle Name Last Name … Answer Miss Bethany Jones … 9021 COMPARE LEFT AND RIGHT What should I see when I click on this link? How do I know which page I’m taken to? How do I know that data loaded that page correctly. If this page isn’t right, the rest of the test cannot proceed. Each cell is a test case. The number of output values to check against will depend on the test case. What other details? Which details are valid details? Which are invalid?
- 19. 19 TEST SPECIFICATIONS Represent the behaviors of the user and the system Created in a simple, human-readable manner Objectively measurable Used as the primary customer and internal acceptance criteria Become the requirements for the development organization Proves delivery
- 20. Create Customer Requirements Create Test Specifications in Gherkin Create Development Design Documents Develop Code Run Regression Tests Multiple Times Daily Prove Delivery Test Specs. Pass. Feature complete, add to regression suite. Unit Testing Manual Testing PROVE DELIVERY • Creating OBJECTIVELY MEASURABLE test specifications • Execute EVERY test cases EVERY day • Measure test coverage against requirements DAILY END RESULTS • Maintain quality • Accurately show % delivery • Foresee Risk
- 21. 21 Execute EVERY Test EVERY Day?
- 22. 22 BUT… Number of Tests 1 VM 1000 8.3 Hours 3000 25 Hours 5000 41.6 Hours 10,000 83.3 Hours Average Test Execution Time: 2 minutes
- 23. 23 THE ANSWER HYPERSCALE ON THE CLOUD
- 24. 24 MAXIMIZE TEST THROUGHPUT THROUGH MASSIVE- SCALE PARALLEL TESTING USING THE CLOUD CLEAR SKY CORE RUNNERS
- 25. 25 SYSTEM CAN’T TAKE THE HEAT? KEEP SCALING! CLEAR SKY CORE RUNNERS LOAD BALANCER
- 26. THE DEMO
- 27. 27 THE CONCLUSION Create unambiguous test specifications, used as requirements Run EVERY test EVERY day to create a safety net Use hyperscaled automation on the cloud to scale Test case Instances under test KEEP QUALITY CONSTANT DAILY