Talk to PS SIG CHI on Agile user research
Download as PPTX, PDF3 likes1,780 views
The document discusses the Rite+Krug method for enhancing stakeholder involvement in agile user research, presented by Jen McGinn and Ana Chang. It combines elements from the Rapid Iterative Testing and Evaluation (RITE) method and Steve Krug's user testing approach to facilitate regular feedback from users and engagement from development teams. Key takeaways include incorporating lightweight research methods, requiring stakeholder attendance, and ensuring consistent user feedback during agile development cycles.
Talk to PS SIG CHI on Agile user research
- 1. The RITE+Krug Method Getting crazy good stakeholder involvement in Agile user research Jen McGinn & Ana Chang December 5, 2013
- 2. Format We’ll tell you our story We’ll listen to your stories We’ll give you summaries and links to further reading Hopefully, you’ll leave with something helpful And we’ll pick up some tricks from you Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 2
- 3. Before we tell you our story, let’s start with you Show of hands: – How many of you are working with Agile development teams? – How many of you are conducting user research (feedback sessions) with those Agile teams? – How many of you are feeling really AWESOME about how the dev team is involved in that research and that they make changes as a result? Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 3
- 4. Late 2011: Traditional usability tests were conducted I can’t believe he clicked the blue button. I assumed he would click the gray button. We should fix that! Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 4
- 5. Later 2011: The stakeholders didn’t buy into the results The results are not statistically significant. This is not a typical user. That’s not a realistic task. I don’t agree with your interpretation of what that means. Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 5
- 6. The feedback came too late for our Agile environment Current state of the application Proposed new design Current state of the application At the beginning of the study Jen McGinn and Ana Chang Start of study When the study results are published PS SIG CHI Monthly Meeting, Dec 2013 6
- 7. Now it’s your turn • First names only • Agile User Research only • In less than a minute, tell us: – What user research you are conducting now (method, frequency, # of participants) – One thing that is AWESOME about that OR – One thing that’s not working, that you wish you could fix Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 7
- 8. Ana had attended a conference Hey, what about RITE testing? Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 8
- 9. Great – so now what? Rapid Iterative Test Environment (RITE) – Developed by Dennis Wixon & the folks at Microsoft games – Can change the (live) system between participants – Developers must attend the UT sessions – Keep testing until show-stoppers are fixed – Number of participants and time to conduct undefined Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 9
- 10. Early 2012: What’s been written on Agile + RITE? • When I started looking for answers 18+ months ago: • Lots and lots on Agile UX + design • Not so much on Agile UX + user feedback • RITE (described in general terms, but no specifics) • Co-design • Customer/user “proxies” • However, there were some elements of Steve Krug’s approach that stuck with me Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 10
- 11. Steve Krug … why does that name sound familiar? http://www.amazon.com/Steve-Krug/e/B001KHCFUU/ 11
- 12. The Krug Approach* “A morning a sprint, that’s all we ask.” • Test 3 users in a morning • Invite everyone to attend: product owners, developers, designers, writers, marketing … • Debrief over lunch: 1 hour – Only people who observed can attend/vote (sort of) – Focus ruthlessly on most serious problems • Report takes less than 30 minutes to write – Just an email – 2 page max., mostly bullet lists * Originally presented by Steve Krug at the Boston UXPA 2013 Conference, May 2013 PS SIG CHI Monthly Meeting, Dec 2013 12
- 13. Our RITE + Krug Combination • Elements from both RITE and KRUG – Require that a stakeholder from dev, design and PM attend each session – Can change the prototype between participants – 3 or 4 participants – Findings discussed in 1-hour debrief – Minimal “report” (bullets on a wiki page) • Sessions went viral: 12+ observers • Could test every as often as every 2 weeks http://www.upassoc.org/upa_publications/jus/2013may/JUS_McGinn_May_2013.pdf Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 13
- 14. Artifacts: scaled down script Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 14
- 15. Artifacts: example wiki page with schedule and impact Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 15
- 16. RITE+Krug Tips & Tricks • Before sessions: Hold a kickoff meeting with stakeholders to get buy-in into the process and time required • During: Keep a running list of observations and email them to the stakeholders within 30 minutes of the last participant • After sessions: Ask attendees of the stakeholder meeting to review your list of observations prior to the meeting • During debrief: Walk through the findings to get agreement on what everyone saw (optional: agree to next steps to fix) • After debrief: Immediately document the observations and list of changes agreed to and send out/post on wiki Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 16
- 17. Let’s talk about Diana The conversation has changed from ‘what is it?’ to ‘how can we master it and make it our own?’ - Diana DeMarco Brown (over lunch one day) http://www.amazon.com/Agile-User-Experience-Design-Practitioners/dp/0124159532/ PS SIG CHI Monthly Meeting, Dec 2013 17
- 18. Diana’s Agile Research Recommendations* • Conduct research before the development cycle if possible – But if you can’t, fit it in anyway • Predictable and repeatable user feedback sessions during the cycle • Complement user feedback sessions with fast, low-cost data collection methods: – Surveys – Customer phone calls – Web analytics – Card sorts – Un-moderated usability tests * Originally presented by Diana Brown at the Boston UXPA 2013 Conference, May 2013 PS SIG CHI Monthly Meeting, Dec 2013 18
- 19. Jeff Gothelf’s Approach • Continuous & collaborative – Continuous: every sprint – Collaborative: conduct research together (dev, design, PM, &research) http://www.amazon.com/Lean-UX-Applying-Principles-Experience/dp/1449311652 Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 19
- 20. Test everything “In order to maintain a regular cadence of user testing, your team must adopt a “test everything” policy. Whatever is ready on testing day is what goes in front of the users… you’ll find yourself taking advantage of your weekly test sessions to get insight at every stage of design and development.” – Lean UX, Jeff Gothelf Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 20
- 21. What I do now: Continuous and Collaborative • Collaborative – Dev, Design, Doc, and PM stakeholders participate or observe – User researcher will facilitate or advise – Your opportunity to get “out of the building” • Continuous – Sessions take place at regular intervals, say once a sprint – User researcher will recruit 3 – 4 target participants per session + debrief same day Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 21
- 22. What else have people written on User Research + Agile? Agile + RITE testing, finally (!) described in detail (2013): • “The RITE Way to Prototype”, Shirey, Charng, & Nguyen http://uxmag.com/articles/the-rite-way-to-prototype Agile + Cognitive walkthroughs (2013): • “Informal cognitive walkthroughs (ICW): paring down and pairing up for an agile world”, Grigoreanu & Mohanna http://dx.doi.org/10.1145/2470654.2466421 Agile + Wizard of Oz testing (2006): • “Adding Usability Testing to an Agile Project”, Meszaros & Aston http://dx.doi.org/10.1109/AGILE.2006.5 Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 22
- 23. Agile & Lean UX resources that include user research • “Adapting Usability Investigations for Agile User-centered Design”, Sy (2007) http://www.upassoc.org/upa_publications/jus/ 2007may/agile-ucd.pdf • “Successful User Experience in an Agile Enterprise Environment”, Federoff & Courage (2009) http://link.springer.com/chapter/10.1007%2F978-3-642-02556-3_27 • Jeff Gothelf’s blog: http://www.jeffgothelf.com/blog/ • 5 tips for applying Lean UX to User Research, Jeff Sauro (2013) http://www.measuringusability.com/blog/lean-research.php Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 23
- 24. Takeaways • You and your team can incorporate user research into Agile development environments • Align research with the Agile development & design cycles • Use lightweight research methods • Use a few participants • Require that key stakeholders attend (and/or conduct) user research • Get feedback at regular intervals (once a sprint) • Get feedback on everything Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 24
- 25. Questions? ? jen dot mcginn at oracle dot com & ana dot ramirez dot chang at oracle dot com Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 25
- 26. The RITE+Krug Method Getting crazy good stakeholder involvement in Agile user research Jen McGinn & Ana Chang December 5, 2013
- 27. Small, think-aloud study Ease of Use Large A/B study Different versions of the UI Jen McGinn and Ana Chang
- 28. A/B testing • A validation technique that you can use when you have: – Working code on the web – Large numbers of participants – A narrowly-defined problem • “Which one of the designs results in fewer abandoned carts?” • “Which one of the designs results in a higher click-through rate?” “Unless you are a huge property like – like Bing or Google … you’ll have more things to test than you have traffic – Lean Analytics, Croll & Yaskovitz, Eric Reis, ed Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 28
- 29. “Formative” Feedback Sessions • A discovery technique that you can use when you have: – drawings, sketches, wireframes – small numbers of participants – questions about features, mental models, terminology, iconography, usability of your prototypes, current practices, pain points, values, design alternatives, or want to collect other qualitative data “Initially, you’re looking for qualitative data … you’re speaking to people … You’re exploring. You’re getting out of the building.” – Lean Analytics, Croll & Yaskovitz, Eric Reis, ed Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 29
Editor's Notes
- #7: A big usability study takes a long time and when you finally have the results, they may no longer be applicable.
- #28: We want to uncover the largest usability problems first, and then we can refine the UI and fix the smaller usability problems. A small, think-aloud usability study will not uncover all usability problems, but it will uncover the largest problems. Many stakeholders think large A/B studies are synonymous with user research. We need to introduce them to other kids of user research.