Testing in a Continuous World
1 like496 views
The document outlines a workshop designed to help participants address challenges in continuous delivery through collaboration and hands-on exercises. Participants engage in identifying real-life challenges, designing experiments, and testing hypotheses related to testing in continuous delivery environments. The use of case studies from two teams illustrates practical approaches to integrating quality assurance and encourages the adoption of effective testing practices.
Testing in a Continuous World
- 1. Welcome to Testing in a Continuous World! Before we start: • Sit with people with different jobs or experience level from yours • Start writing down your challenges around continuous delivery, one per sticky note (more time for this later) @LISACRISPIN | @LISIHOCKE
- 2. Testing in a Continuous World HOW TO CONTINUOUSLY LEARN FOR SUCCESSFUL CONTINUOUS DELIVERY @LISACRISPIN | @LISIHOCKE
- 3. Introduction SETTING THE STAGE @LISACRISPIN | @LISIHOCKE
- 4. Setting expectations WHAT WE WON’T DO • Tell you the one and only solution • Provide a blueprint to copy • Lecture you WHAT WE WILL DO INSTEAD • Give some inspiration what you could try • Provide a tool you can use after you returned to work • Facilitate a hands-on, interactive and collaborative exercise @LISACRISPIN | @LISIHOCKE
- 5. 1. Identify real-life challenges for testing in a continuous delivery environment 2. Design real-life experiments to address those challenges 3. Take the experiments and try them at your workplace Learning intentions @LISACRISPIN | @LISIHOCKE
- 6. Continuous delivery • Release software reliably and without friction (Martin Fowler) • Production deploys should be boring (Jez Humble) • How can we feel confidence for frequent deploys? @LISACRISPIN | @LISIHOCKE
- 7. Real-life stories • Real teams; in the following named Team Orange & Team Blue • Real challenges • Real hypotheses • Real experiments • Real results @LISACRISPIN | @LISIHOCKE
- 8. Team Orange: Challenge Challenge: The team has a perception that quality can be assured by one person at the end @LISACRISPIN | @LISIHOCKE
- 9. Team Orange: Hypothesis I believe that renaming the “QA Review” column to “Final Testing” and breaking out of the perceived role limits will result in the team’s change of perception from assuring quality towards building quality in by the whole team, and from testing late towards testing throughout the workflow. I’ll know I have succeeded when I don’t hear about quality assurance anymore or get questioned why starting testing early after six months. @LISACRISPIN | @LISIHOCKE
- 10. Team Orange: Experiment • Rename column to “Final testing” to reflect only last checks on story level before merging • Start testing as early as possible, test incrementally, also on production • Role rebranding: changed title from “QA Developer” to “Agile Tester” @LISACRISPIN | @LISIHOCKE
- 11. Team Orange: Results • Team started talking about testing as activity instead of QA as phase or role • Team stopped questioning testing throughout the workflow • Team appreciated early feedback • Increased transparency of board @LISACRISPIN | @LISIHOCKE
- 12. Team Blue: Challenge Challenge: • Many regressions after move to twice-weekly deploys • Still had manual release regression checklists • Many release-stopper bugs, bad regressions getting into production @LISACRISPIN | @LISIHOCKE
- 13. Team Blue: Hypothesis We believe that pairing in exploratory testing and automation will result in a reduced frequency of production release rollbacks and hotfixes. We’ll know we succeeded when the average number of release rollbacks and hotfixes decreases from 5 per month to 3 per month within 3 months. @LISACRISPIN | @LISIHOCKE
- 14. Team Blue: Experiment • Stop doing manual release regression testing • Developer pairs do exploratory testing on each story • Testers pair at least 1 day per week with developers @LISACRISPIN | @LISIHOCKE
- 15. Team Blue: Results • Average went down to 3 per month for 3 months • But has crept back up to 5 • Many factors involved • Tester/developer pairing now established practice, but trying more experiments @LISACRISPIN | @LISIHOCKE
- 16. How to test continuously, successfully? What is key: • Visibility • Shared understanding • Short feedback loops → Let’s look at just a few practices that can help @LISACRISPIN | @LISIHOCKE
- 17. Visibility Some visualization techniques: • Draw and discuss together • Mind maps • System architecture • Workflow & pipelines @LISACRISPIN | @LISIHOCKE
- 18. Shared understanding Some framework examples: • Behavior-Driven Development • Example mapping • Specification workshops • User story mapping @LISACRISPIN | @LISIHOCKE
- 19. Short feedback loops FROM CLOSE COLLABORATION • Stop starting, start finishing • Cross-functional pairing • Mobbing FROM TESTS • TDD • Adequate automated test coverage • Exploratory testing @LISACRISPIN | @LISIHOCKE FROM USERS • Feature toggles • Monitoring in production • Analytics • Direct feedback FROM INFRASTRUCTURE • Continuous integration • Spin up test environments • Deploy automatically on green build
- 20. Exercise GROUP WORK IN SIX ACTS @LISACRISPIN | @LISIHOCKE
- 21. Exercise Part 1: Brainstorm challenges @LISACRISPIN | @LISIHOCKE
- 22. Exercise Part 1: Brainstorm challenges 1. Write down your challenges around continuous delivery 2. One challenge per sticky note @LISACRISPIN | @LISIHOCKE
- 23. Exercise Part 2: Choose a challenge @LISACRISPIN | @LISIHOCKE
- 24. Exercise Part 2: Choose a challenge A bit like Lean Coffee: • Go around the table, each person gets 30 seconds to explain their challenge to others in the group • Group similar topics together • After everyone has explained their topics, dot vote to choose a challenge to work on – 3 votes per person • Prioritize topics by votes • Start with the top challenge (if time is left you can work on more than one) @LISACRISPIN | @LISIHOCKE
- 25. Exercise Part 3: Design your experiment! @LISACRISPIN | @LISIHOCKE
- 26. Exercise Part 3: Design your experiment! For your chosen challenge: • Create a measurable hypothesis with a prediction • Design an experiment to test the hypothesis in the chosen work context (be specific) • Make a poster showing challenge, hypothesis & experiment Challenge: ______________________ Hypothesis: • We believe that __________________ • will result in _____________________ • We’ll know we have succeeded when _____________________________ Experiment: • _____________________________ • _____________________________ @LISACRISPIN | @LISIHOCKE
- 27. Exercise Part 4: World Café @LISACRISPIN | @LISIHOCKE
- 28. • Choose a spokesperson to take your poster and travel to the next table on the left • Everyone else stays at their table • Spokespeople explain the challenge, hypothesis and experiment to the visited group • The group gives feedback and contributes their ideas for the presented experiment Exercise Part 4: World Café @LISACRISPIN | @LISIHOCKE
- 29. Exercise Part 5: Incorporate feedback @LISACRISPIN | @LISIHOCKE
- 30. Exercise Part 5: Incorporate feedback • Tweak your hypothesis and experiment based on the feedback you got from others • Make a new poster if needed • Get ready to present your challenge, hypothesis and experiment to everyone ☺ @LISACRISPIN | @LISIHOCKE
- 31. Exercise Part 6: Group presentations @LISACRISPIN | @LISIHOCKE
- 32. Exercise Part 6: Group presentations You have 1min per group to present… • the challenge, • the measurable hypothesis, • the designed experiment, • the most valuable idea you got from the other groups that you decided to incorporate, • and who will run the experiment to test the hypothesis back at work! @LISACRISPIN | @LISIHOCKE
- 33. Closing WRAP-UP. QUESTIONS? GIVEAWAYS! EVALUATIONS @LISACRISPIN | @LISIHOCKE
- 34. 1. Identify real-life challenges for testing in a continuous delivery environment 2. Design real-life experiments to address those challenges 3. Take the experiments and try them at your workplace Learning intentions reviewed @LISACRISPIN | @LISIHOCKE
- 36. Save 35%* off the books or ebooks eBook formats include EPUB, MOBI, & PDF See www.agiletester.ca for more Save 50%* on Agile Testing Essentials LiveLessons Video Training Use code AGILETESTING at informit.com/agile *Discount taken off list price. Offer only good at informit.com and is subject to change. @LISACRISPIN | @LISIHOCKE