Big Visible Testing

Editor's Notes

• #2: Hi, I'm Claire Moss and I'm here to talk to you about Big Visible Testing. For those of you who like to tweet, you can see my Twitter handle @aclairefication here in the header and my talk's hash tag #BVT along with the conference hash tag #CAST2012.I have always thought of myself as an agile tester - hence my talk's riff on the agile Big Visible Charts concept. However, over the last year, I have been transforming into a truly agile tester working as part of a Scrum team to build a nearly green field product.
• #3: The other members of my product delivery team are from a mix of disciplines but do not have a background in software testing as I know it. This audience for my testing needs transparent, quick information access in a form that best suits their distinct perspectives. It has been a learning curve for all of us and I hope that you benefit from our experience. I hope you will leave this presentation with a fresh perspective on your product team members so that you can focus on the value of the testing artifacts you produce.
• #4: One of the Scrum artifacts that we produced for our product team is the Definition of Done. As you can see, we define the results we want for each story with an emphasis on quality, so the testing information needs to support this value. You may notice that the product team isn't interested in the overall picture of the testing process in the way my testing manager is. They want more short-term feedback from testing so that they can do something about it.
• #5: My first attempt to educate my teammates about testing was to create a low-tech defect dashboard, or as I like to call it the bug board. I started with defects because my teammates already had some understanding of the relationship between software development and that form of testing feedback. I wanted to meet them where they were and to collaborate on the testing reporting they needed to schedule, design, and implement stories.
• #6: Imagine my delight the morning I arrived at work to find my bug board rearranged: it had been the focus of fruitful discussion among the members of my Product Owner team, that is the folks making the decisions about what to do.
• #7: As our pilot release progressed, we had deeper discussions about how testing serves the product team and I gained insight into their various disciplines, ways of thinking, and preferred ways to communicate. After we reflected on what we learned, I resolved to apply this to future experiments.
• #8: We iterated and I came to see my teammates as quite distinct customers of my work. I began to think of them in the way they think of their customers, applying the lessons they taught me to them(selves?). And so I developed user personas corresponding to what value each of them emphasized. Since these personas are quite detailed, I will only be able to hit the highlights in this presentation but will be glad to discuss more afterward.I began by focusing on the goals and greatest concerns for each of these roles. Since user personas can be proposed ad hoc and evaluated through research, I began with a guess based on my past experience and tested that against my current context.
• #9: [READ ABOUT BEN THE BUSINESS ADVOCATE]
• #10: For Ben, I chunked bugs by Severity and discussed that with him, aiming for ease of reference.
• #11: We revised the representation so that he could efficiently gather information and at a glance know when to follow up with questions to elicit well-rounded information for making business decisions. He wanted to know whether to address defects and when we needed to complete them. When our critical and highest severity buckets were empty, he thinks, "Let's get a return on our investment. Ship it!“Although Ben is the CEO of the product, his is not the only perspective to consider. There is an inherent tension between this business motivation and what drives my second persona Ursula.
• #12: [SHOW URSULA]She balances the team by bringing a sharp focus on the users, whom she knows deeply through broad customer contact, whether through usability testing or contextual inquiry. She sees the system through their eyes - and frequently prefers a visual representation of the impact on the user.[READ ABOUT URSULA THE USER ADVOCATE]
• #13: For Ursula, I organized bugs around a site map I built for our web application. Ursula is a "doer" since she designs the user experience. In Scrum terms, she is a pig. In UX terms, she is a primary persona: I want to optimize my quality information for her use in problem resolution.Since Ursula is also a judicious perfectionist with a focus on consistency, she and I have a great relationship and collaborate closely. I use her designs and user research to feed my test design and she uses my granular test output in the larger context of the product.
• #14: Ursula consumes defects more in clusters than singly, so we revised the bug board accordingly. I can enhance the bug board even more by focusing on efficient translation to her action. Thus describing problems as a scenario emphasizes when and how bugs impact a user's day-to-day life with the software so she can iterate on the user experience, which encompasses more than just graphic design. In turn, I can provide feedback and critique mockups or prototypes, asking helpful questions.
• #15: While everyone on the team is conversant with software from our own usage, no one has as deep an understanding as Todd the Technologist - sometimes he has to explain his recursion jokes for the rest of us.[READ ABOUT TODD THE TECHNOLOGIST]As the principle "doer" on the team, he is my highest priority customer. Although Ursula has the chops to get into the user interface code, Todd is the one to keep things running smoothly behind the scenes. Test-driven development and automated checks are his first line of defense again bugs, so he may respond with puzzlement to the presence of a bug backlog. Let's translate it for him.
• #16: First, we tried scheduling a time box for bug fixing, our bug bucket, to attack our most urgent concerns since Todd cares about what is the highest priority work so he can attack it. However, Todd is not interested only in an expedient solution. He wants to keep the product sustainable, so he may report issues for the backlog that incorporate technical debt, which may relate to clustering of defects.
• #17: Ursula isn't the only one who likes a good user scenario, so we shifted to relating bugs to stories scheduled for sprints. After all, Todd needs to know the whole picture so he can incorporate refactoring and bug fixing in the estimate along with the feature enhancement that together produce value for the user. He also likes preventative measures such as a scripted smoke test that covers a story's acceptance criteria - just another way for him to know that a story is done.
• #18: That brings us full circle to Ben who makes the ultimate judgment on whether the story is done when he runs the application against the acceptance criteria. Although Ben may be a secondary persona for me - one I want to accommodate but not as directly as my primaries - he benefits from all the testing information along the way since he has confidence when he accepts a story.Our current sprint board informs Ben about the status of our sprint's story progress, including defect discovery.Now that the whole team understands defects better, we also have begun to incorporate exploratory testing charters in our stories so that everyone can see the progress of testing over time and what charters uncover the defects. I have been able to involve everyone on the team in exploratory testing to some degree and our product's users will reap the benefit of the common understanding. After all, while I may be in the information business as a tester, it is all ultimately in service of our users.
• #19: I want to thank Will & Astrid for teaching me all about user personas and the role that UX plays in software development, but I especially want to thank my employer Daxko for sending me to CAST so we can confer today. And thank you my audience for your time.Any questions from those present or anyone following along at home?