Seven Keys to Navigating Your Agile Testing Transition
- 1. MF AM Tutorial 4/29/13 8:30AM Seven Keys to Navigating Your Agile Testing Transition Presented by: Bob Galen RGalen Consulting Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
- 2. Bob Galen Bob Galen is an agile coach at RGalen Consulting and director of agile solutions at Zenergy Technologies, a North Carolina-based firm specializing in agile testing and leading agile adoption initiatives. Bob regularly speaks at international conferences and professional groups on topics related to software development, project management, software testing, and team leadership. He is a Certified Scrum Master Practicing (CSC), Certified Scrum Product Owner (CSPO), and an active member of the Agile Alliance and Scrum Alliance. Bob published Scrum Product Ownership–Balancing Value from the Inside Out, which addresses the gap in guidance toward effective agile product management. Contact Bob at bob@rgalen.com or bob.galen@zenergytechnologies.com.
- 3. Keys for Transitioning to Agile Testing Myths & Realities from the Trenches Bob Galen President & Principal Consultant RGCG, LLC bob@rgalen.com Fifteen Seven Keys for Navigating your Agile Testing Transition Myths & Realities from the Trenches Bob Galen – RGCG bob@rgalen.com Mary Thorn – Deutsche Bank mary.thorn@gmail.com 1
- 4. Introduction Bob Galen Somewhere ‘north’ of 30 years experience ☺ Various lifecycles – Waterfall variants, RUP, Agile, Chaos Various domains – SaaS, Medical, Financial Services, Computer & Storage Systems, eCommerce, and Telecommunications Developer first, then Project Management / Leadership, then Testing Leveraged ‘pieces’ of Scrum in late 90’s; before ‘agile’ was ‘Agile’ Agility @ Lucent in 2000 – 2001 using Extreme Programming Formally using Scrum since 2000 Currently an independent Agile Coach (CSC – Certified Scrum Coach, one of 50 world-wide; 20+ in North America) at RGCG, LLC and Director of Agile Solutions at Zenergy Technologies From Cary, North Carolina Connect w/ me via LinkedIn and Twitter if you wish Bias Disclaimer: Agile is THE BEST Methodology for Software Development However, NOT a Silver Bullet! Copyright © 2013 RGCG, LLC 3 Introduction Mary Thorn A VP of QA and Test Manager at Deutsche Bank Global Technologies in Cary, North Carolina, Mary Thorn has a broad testing background that spans automation, data warehouses, and web-based systems in a wide variety of technologies and testing techniques. During her more than fifteen years of experience in healthcare, HR, agriculture, and SaaS-based products, Mary has held manager and contributor level positions in software development organizations. She has a strong interest in agile testing methodologies and direct experience leading agile teams through Scrum adoption & beyond. Copyright © 2013 RGCG, LLC 4 2
- 5. Outline – Myths & Realities Introduction 1. Transforming your Team 2. Automation 3. Developers & Automation 4. Developers Testing 5. Test Planning & Scripts 6. Testing within the Sprint 7. Exploratory Testing 8. Role of Testers 9. Developer to Tester Workflow 10. Managing Agile Testers 11. Test Metrics 12. Retrospectives – The Secret Sauce 13. Continuous Improvement 14. The Customer 15. Agile Requirements – The Product Backlog Copyright © 2013 RGCG, LLC 5 #1, Transforming your team Myth: You need all programmers or highly technical testers when you move to agile Reality: A mix is best – Manual, domain-centric and technical skills Some programming / scripting skills Soft / collaborative skills Reality: And throw out all of that Developer-to-Tester ratio ‘stuff’. Copyright © 2013 RGCG, LLC 6 3
- 6. #2, Automation Myth: You need 100% automation to start agile testing. Reality: You simply need to have a strategy AND doggedly pursue automation where it makes sense Make it part of the Backlog and work it every sprint Reality: There are some excellent Open Source tools that supplement agile automation development Reality: The Agile Test Automation Pyramid is the right overall strategy Copyright © 2013 RGCG, LLC 7 Agile Test Automation Pyramid Mike Cohn; Lisa Crispin & Janet Gregory http://behaviordrivendevelopment.wikispaces.com/Testing Copyright © 2013 RGCG, LLC 8 4
- 7. #3, Developers & Automation Myth: QA designs, writes & runs all of the test automation Reality: Everyone should be responsible for automation Developers need to minimally attend to Unit Level Participate in any framework or re-use development Writing ‘glue’ code – fixtures, step files, etc. Reality: It also extends into your Build & Continuous Integration systems All automation should be ‘wired’ into CI Dashboards, trending, lava lamps, etc. for all to see Copyright © 2013 RGCG, LLC 9 #4, Developers Testing Myth: Developers can’t test their own code—they’re not independent enough nor skilled enough to do it properly. Reality: We need to stop stereotyping team members, their strengths and their abilities. Developers can absolutely test their own code. Some are better at it than others Pair with them to help test appropriately Copyright © 2013 RGCG, LLC 10 5
- 8. #5, Test Planning & Scripts Myth: You don’t need to plan (it just happens ) and you don’t need functional test cases (automation takes care of everything ) Reality: Plans help the team focus on the risk-based testing required within an iteration AND across a release Reality: Scripts (test cases) help formalize and drive your testing; Absolutely required in regulatory environments Reality: You’ll never actually automate every test Copyright © 2013 RGCG, LLC 11 #6, Testing within the Sprint Myth: You simply need to run 100% of the tests within the constraints of the Sprint “Agile” that’s Reality: Rarely possible in most contexts. You first need a high-degree of automation and business support (for example: equipment costs) Very mature test automation and CI / CD environments Reality: Most agile teams adopt some sort of risk-based testing approach for within the sprints Dealing with Technical Test Debt Then leverage Hardening / Stabilization pre-release sprints Copyright © 2013 RGCG, LLC 12 6
- 9. The Agile Release Train Synchronized Internal Release Team 1 Iterate Iterate Harden Continuous Team 2 Iterate Iterate Harden Continuous Team 3 Iterate Iterate Harden Continuous Team 4 Iterate Iterate Harden Continuous External Release Iterate Iterate Iterate Integration Docs, Training, X-team Support, Integration Harden UAT, Comp. Iterate Iterate Iterate Iterate Iterate Iterate Integration Iterate Iterate Iterate Integration Team n Copyright © 2013 RGCG, LLC 13 The Agile Release Train Example: eCommerce / SaaS Model 10 days Team 1 10 days Iterate Iterate 5 + 2 days External Release Harden Continuous Integration Team 2 Iterate Iterate Harden X-team Harden Rinse Harden Docs, Training Repeat Continuous Integration Team 3 Iterate Iterate & Continuous Integration Team 4 Team 8 Iterate Iterate Harden Continuous Integration Environment Evolution Dev + QA Copyright © 2013 RGCG, LLC Dev + QA QA -> Staging Production 14 7
- 10. The Agile Release Train Example: iContact / SaaS Model 3 weeks / 15 days SBET, Exploratory – Regression Testing 4-5 days Production Release Team 1 Iterate Harden Continuous Integration Team 2 Iterate Harden X-team Harden Continuous Integration Team 3 Iterate Harden Docs, Training Rinse & Repeat Continuous Integration Team 4 Team 10 Iterate Harden Continuous Integration Environment Evolution Dev + QA QA -> Staging Production Copyright © 2013 RGCG, LLC 15 Brainstorm… “Your” Agile Release Train Either individually or in small groups from the same company Take a few minutes and think about your current release constraints timing, customers, domain, competition, # of teams, technology, etc. Design a release train model for your organization Overlay it with testing activities, plans, and milestones Present it to your larger table/group; gain feedback & adjust Be prepared to share Copyright © 2013 RGCG, LLC 16 8
- 11. #7, Exploratory Testing Myth: There is no place for Session Based Exploratory Testing in agile contexts. Reality: ET and SBET are a beautiful complement to agile testing. Helping nurture pairing & collaboration across teams and functions Defining new (more valuable) test cases Quickly gaining quality & usability feedback Copyright © 2013 RGCG, LLC 17 #8, Role of Testers Myth: That the testers alone own quality & testing practices within each team and sprint Reality: The testers foster a “Whole Team” view towards quality—focusing less on “Testing” and more on “Quality Practices & the Customer” Serving as guides for the team; Testing the “hard bits” Facilitating exploratory testing sessions—finding more interesting / valuable tests Working with the Product Owners—are we solving the customers problems? Copyright © 2013 RGCG, LLC 18 9
- 12. #9, Developer to Tester Workflow Myth: There is always a hand-off from developers to testers; usually quite late in the sprint. That’s simply the “way of things” in software development. Reality: Scrummer-fall is alive and well but, Wrong! Teams need to swarm on their work, as flow & throughput matter the most. WIP limits and close proximity / collaboration help establish a healthy tempo of developer & tester pairing Micro-handoffs – testing as development progresses! Do you log bugs? Or do you fix bugs? Copyright © 2013 RGCG, LLC 19 #10, Managing Agile Testers Myth: The functional test manager is in charge of deciding how, who, when , etc. for the test team. Reality: You still absolutely need functional leadership within agile teams; However, it’s focused towards quality practices, strategy & coaching, and handling impediments / escalations Encouraging transparency, transforming metrics & reporting Supporting & protecting the teams Encouraging risk-taking, innovation & creativity (Slack Time) Copyright © 2013 RGCG, LLC 20 10
- 13. Levels of Done-Ness Criteria Activity Criteria Basic Team Work Products Done’ness criteria User Story or Theme Level Acceptance Tests Sprint or Iteration Level Done’ness criteria Release Level Release criteria Example Pairing or pair inspections of code prior to check-in; or development, execution and passing of unit tests. Development of FitNesse based acceptance tests with the customer AND their successful execution and passing. Developed toward individual stories and/or themes for sets of stories. Defining a Sprint Goal that clarifies the feature development and all external dependencies associcated with a sprint. Defining a broad set of conditions (artifacts, testing activities or coverage levels, results/metrics, collaboration with other groups, meeting compliance levels, etc.) that IF MET would mean the release could occur. Copyright © 2013 RGCG, LLC 21 #11, Test Metrics Myth: You can and should move forward reporting everything exactly as you have before. Including any ‘dysfunctional’ metrics that your process and/or PMO dictates. Reality: The metrics should change immediately. From QA and Test centric towards Team-Centric metrics (Value, Throughput, Quality, Team) Stop reporting out on “Testing”; it’s irrelevant! This effects planning as well—estimation, progress, risk, etc. Contribute quality-centric Information radiators to the mix Copyright © 2013 RGCG, LLC 22 11
- 14. Brainstorm… Morphing your Metrics Get together in small groups of of 4-6. What are you measuring today? Why? How are they driving your success and behaviors? As you move to agile, what can/should you be measuring in the 4 key areas: Value, Quality, Throughput & Predictability, and Team How will you change your existing metrics? What behaviors are you trying to inspire? Copyright © 2013 RGCG, LLC 23 #12, Retrospectives: The Secret Sauce Myth: Testers are “Second Class” citizens who don’t play an active part in the project & team Reality: There are many places to “make a difference” Getting the 800 lb. Gorillas out on the table; Showing courage; telling truth Fostering continuous improvement within the team Setting the example; showing vulnerability—admitting you’re wrong Team listening; active planning; dependencies; pairing Risk-taking; Failure! Copyright © 2013 RGCG, LLC 24 12
- 15. #13, Continuous Improvement Myth: We’re generally ‘stuck’ in our approaches so just accept them and do the “best you can”. Reality: Continuous improvement is everyone’s responsibility—to engage, suggest, take ownership of current results, explore root causes, etc. Active participation in your teams Retrospectives is a key way to guide quality, testing, and customer-centric improvements. Courage! Copyright © 2013 RGCG, LLC 25 #14, The Customer Myth: Business Analysts capture customer requirements and testers test them for completeness. Reality: You need to begin to partner with the Customer – Stakeholders – Product Owners to produce software that solves the their problems. Move to the “front” and help define & refine User Stories with your Product Owner Actively participate in Sprint Reviews Show value for automation; placing test investments in the Backlog Copyright © 2013 RGCG, LLC 26 13
- 16. #15, Agile Requirements – The Product Backlog Myth: We can’t start testing until the requirements are finished or stable; no matter how ‘agile’ we are. Reality: Hogwash! Get over it Ambiguity and incompleteness need to become your friend and ally. As does working with your Product Owners and Customers to help define the requirements Realizing that the requirements (User Stories) are only complete at the end of each sprint. Copyright © 2013 RGCG, LLC 27 Agile Test Transformation Strategy: 3 Pillars of Agile Quality Copyright © 2013 RGCG, LLC 28 14
- 17. 3 Pillars of Agile Quality Development & Test Automation • Pyramid-based Strategy: (Unit + Cucumber + Selenium) Software Testing • Risk-based testing: Functional & NonFunctional • • • Test planning @ Release & Sprint levels • Exploratory Testing • Standards – checklists, templates, repositories • Balanced across manual, exploratory & automation • • • Agile-centric Metrics Continuous Integration Attack legacy technical debt in the Backlog Visual Feedback – Dashboards Actively practice ATDD and BDD Cross-Functional Team Practices • Team-based Pairing • Stop-the-Line • Code Reviews & Standards • • Active Done-Ness • Aggressive Refactoring • User Stories – 3 Amigo based Conversations • Building the ‘Right’ Solutions 29 Copyright © 2013 RGCG, LLC 3 Pillars of Agile Quality Development & Test Automation • Pyramid-based Strategy: (Unit + Cucumber + Selenium) • Continuous Integration • Attack legacy technical debt in the Backlog • Visual Feedback – Dashboards • Actively practice ATDD and BDD A central part of agile adoption is focusing on CI, 3tiered automation development, and Dashboards to begin incrementally building coverage for faster feedback on changes. In the interim, Hardening or Stabilization Sprints and having a risk-based Release Train concept help It’s important that Test or QA not ‘own’ the tooling or all of the automation efforts. The strategy can come from Test, but the automation development is best left to the team. Mature teams invest in automation as part of Doneness and continually on their backlogs 30 Copyright © 2013 RGCG, LLC 15
- 18. 3 Pillars of Agile Quality Software Testing • Risk-based testing: Functional & NonFunctional • Test planning @ Release & Sprint levels • Exploratory Testing • Standards – checklists, templates, repositories • Balanced across manual, exploratory & automation • Agile-centric Metrics Exploratory Testing (Charter / Session based and paired) can be an incredibly effective way to establish a whole-team, collaborative view towards quality and testing. It also emerges new tests. Leverage ‘plans’ as a whole-team collaboration mechanism and do plan. Do not measure testing or tester progress; instead, measure throughput, output, sprint outcomes, and done-ness escapes at a team level. You need a balanced test team; not everyone needs to be able to program. But everyone needs to be skilled testers. Agile testing is a Risk-Based play in every Sprint and across a release sequence. Don’t forget your techniques! 31 Copyright © 2013 RGCG, LLC 3 Pillars of Agile Quality Cross-Functional Team Practices • Team-based Pairing • Stop-the-Line • Code Reviews & Standards • • Active Done-Ness • Aggressive Refactoring • User Stories – 3 Amigo based Conversations • Building the ‘Right’ Solutions One of the hardest areas to get ‘right’ culturally. It needs leadership alignment from Quality/Testing to Product to Development and a consistent voice of whole-team approaches. This is where LEAN lives, where whole-team collaboration happens, where professionalism and craftsmanship are held dear. I like the view of testers becoming the VOC, champions of quality, and consistent questioners of what is being build. Are we solving the right problems as simply as possible. MMF, MVP, MMP, etc. And yes Virginia, there ARE standards and consistency! 32 Copyright © 2013 RGCG, LLC 16
- 19. Software Testing Strategies It ALL starts with empowering testers AND creating a Whole-Team view towards Quality Critical Early Steps: Creating a sense of empowered Functional Team Applying Testing Standards across all teams Deploying Exploratory Testing across all teams Defining a core set of Agile KPI / metrics ACTIVE participants in Sprint Planning Copyright © 2013 RGCG, LLC 33 Cross-Functional Team Practices Strategies Training Agile / Lean in general, Story writing, Acceptance, Unit testing, etc. Teaming – for example: feedback or 5 Dysfunctions / Trust Critical Early Steps: Coaches & Scrum Masters to reinforce: Pairing / Swarming; WIP Limits across teams Define prescriptive and aggressive Done-Ness for ALL teams Implement coding standards & Crucible / code reviews across the center (appropriate for technology stacks) Release Planning BEFORE allowing a team to start Sprint #1 Backlogs have Bug + Refactoring + Automation targets (20%)? Copyright © 2013 RGCG, LLC 34 17
- 20. Organizational Quality Strategies Continuously communicate your unified Vision Your strategy must be aligned/shared across: Development, Quality/Testing, and Product Keep working your strategy across the pillars Don’t get stuck with too narrow a focus (easy road) Make your strategy visible (Information Radiators) Show progress (Ex: burn up of test automation coverage across tiers) Visualize organizational impediments to your Agile Quality strategies Attack them! Quarterly read-outs on progress, plans and adjustments Listen to your teams Celebrate successes! Copyright © 2013 RGCG, LLC 35 What will be (your) agile strategy when you get back home? Either in groups or individually Consider the 3-Pillars discussion Consider your current team / organization agile ‘state’ Define a broad, 3-pillar view towards some immediate focus points when you get back into the office What will you focus on? Why? How will you communicate the need for change? How will you measure results? What will come immediately afterwards? Copyright © 2013 RGCG, LLC 36 18
- 21. Wrapping up… Agile is the best thing that’s happened to testers since The Great Depression Whole Team view Testing, Metrics, Automation Planning, Reporting, Quality Facilitate feedback Multi-tiered automation Just-in-Time, risk-based testing Continuous improvement Trust the Team Retrospective Copyright © 2013 RGCG, LLC 37 Contact Info Bob Galen Principal Consultant, RGalen Consulting Group, L.L.C. Director of Agile Solutions, Zenergy Technologies, Blogs Project Times http://www.projecttimes.com/robert-galen/ Business Analyst – BA Times http://www.batimes.com/robert-galen/ My Podcast on all things ‘agile’ http://www.meta-cast.com/ Experience-driven agile focused training, coaching & consulting Contact: (919) 272-0719 bob@rgalen.com bob.galen@zenergytechnologies.com www.rgalen.com Copyright © 2013 RGCG, LLC 38 19
- 22. Additional Topics Copyright © 2013 RGCG, LLC 39 Two Pillars of Lean ‘Thinking’ Respect for People Customer, Employees, Vendors Develop your teams Trust & coach No wasteful work Continuous Improvement Embrace change, challenge everything Kaizen – small, incremental change Kaikaku – larger scale, fundamental From http://www.leanprimer.com Copyright © 2013 RGCG, LLC 40 20
- 23. Agile Testing Quadrants Brian Marick; Lisa Crispin & Janet Gregory Business Facing Functional tests Story tests Examples Prototypes Simulations Exploratory testing Scenarios Usability testing UAT Alpha / Beta Q2 Q3 Q1 Q4 Unit tests Component tests API tests Automated & Manual Manual Performance testing Load testing Security testing Non-functional requirements Technology Facing Critique the Product Supporting the Team Automated & Manual Automation, Tools, and Manual Copyright © 2013 RGCG, LLC 10 Tenets of Agile Testing 41 Jean Tabaka, Rally Software 1. The system always runs Continuous Integration 2. Stop the line, vs. logging defects Lean – fix it now! 3. If it’s not tested, it’s not “Done” Early feedback; Earned Value 4. Testing comes first, not last Collaborative testing, focus on building in quality 5. Finding defects after Development is “Done” is too late Early feedback; fix it now! Copyright © 2013 RGCG, LLC 42 21
- 24. 10 Tenets of Agile Testing Jean Tabaka, Rally Software 6. “Development Complete” is meaningless Whole Team complete view – no “partial credit” 7. Use testing, not analysis, to explore requirements Executable requirements 8. Automation is “how” not a “whether” or “when” Automate all testing; feedback 9. Tests are your second most detailed specification Code is first; later is traditional specifications 10. Testers are CustomerDeveloper liaisons VOC; guide effective team collaboration; ask the right questions Copyright © 2013 RGCG, LLC 10 Commitments of Agile Testing 43 Jean Tabaka, Rally Software 1. We commit to not moving forward if a hole is found through root cause analysis without first writing a test 2. We commit to not relying solely on just automated testing or just manual testing 3. We commit to not sitting behind a QA wall (no boundaries!) 4. We commit to not allowing a code complete without test code harness complete 5. We commit to not waiting for a test phase but rather working in smaller and smaller pieces, sooner and sooner Copyright © 2013 RGCG, LLC 44 22
- 25. 10 Commitments of Agile Testing Jean Tabaka, Rally Software 6. We commit to not testing one iteration after development is “Done” 7. We commit to not allowing surprises to accumulate for large end-to-end testing (“mock it now”) 8. We commit to not leaving the riskiest tests to the end 9. We commit to being an equal participant with the customer and the developer in defining “Doneness” 10. We commit to not taking this oath lightly Copyright © 2013 RGCG, LLC 45 23