Harvesting user insights revolve conf v09
1 like1,480 views
The document from the 2017 Revolve Conference discusses the importance of user insights in product design, emphasizing that successful product creation relies on understanding real users' needs and behaviors. It outlines different techniques for gathering user insights, including ethnographic methods and the talk-aloud protocol, which helps to uncover users' thoughts and experiences during interactions with products. The presentation also highlights the necessity of involving the entire product team in researching and verifying users' truths to avoid biases in the design process.
Harvesting user insights revolve conf v09
- 1. Revolve Conference 2017 – Charleston, SC darrenkall@specificclarity.com @darrenkall Darren Kall – Managing Director – Specific Clarity Harvesting User Insights from Real Users in the Real World © Specific Clarity 2017
- 3. Time breakdown • ~15 min. Value of user insights • ~15 min. Description of technique • ~30 min. Demos and practice 3
- 4. Product = 4 Products digital and physical Service System and process Concepts and ideas Spaces Etc. App or device Workflow, task flow Marketing campaign Strategic plan Museum exhibit Movie or video Presentation Etc.
- 5. Product = 5 Any “thing” you make that a human being interacts with
- 6. User = 6 Buyer or end-user Passive or active Maker or consumer Observer or operator Etc. Customer or employee Administrator or installer Parent or child Passenger or driver Theater goer or patient Tenant or visitor Etc.
- 7. User = 7 Any human being that interacts with any part of the product in its end to end life cycle
- 8. Successful product creation depends on creating customer value 8
- 9. To create customer value you must have user insights 9
- 10. 10
- 11. Intuitive design genius who knows all things about all people 11 Option 1:
- 13. Your task is to build a calendar & scheduling app 13
- 14. Those two people, that you cannot see, are your customers 14
- 15. What do you know about them and what they need from a calendar app? 15
- 16. Set up notes Two columns 16
- 17. Start with demographics When you guess, shout out “I’m guessing” 17
- 18. What are their names? 18
- 19. How tall are they? 19
- 20. What were they wearing? 20
- 23. Do they have kids? How old are the kids? 23
- 24. What do they do for a living? 24
- 25. Do they rent or own where they live? 25
- 26. What leisure activities do they enjoy? 26
- 27. What groups do they belong to? 27
- 28. What company marketing segment do they belong to? • Traditionalists: 1900 –1945 • Baby Boomers: 1946 –1964 • Gen X: 1965 –1976 • Gen Y: Millennials 1976 – 1995 • Gen Z: 1995- now 28
- 29. Forgive me … 29
- 30. How does any of this relate to their calendar app needs? What did their demographics teach you about their calendar behaviors, needs, expectations, or scenarios of use? 30
- 31. Specific calendar app questions 31
- 32. • How many calendars do they each have? • What devices do they interact with their calendars on? • How many other people interact with their calendars? • Do they manage the calendar for someone else? • What capabilities do they expect for sharing calendars? • Those kids, if they have them, do they play soccer? • Is one of them running the calendar for the kid’s team? • Do they want their work and home calendar separate? • How do they correct mistakes in their calendar? • How do they accept new invitations in their calendar? • Etc. 32
- 33. You guess a lot! Let’s have them back in and see how well you did. 33
- 34. • What are their names? • How tall are they? • What were they wearing? • Glasses? Beards? Jewelry? • Are they married? • Do they have kids? • How old are the kids? • What do they do for a living? • Do they rent or own where they live? • What leisure activities do they enjoy? • What groups do they belong to? • What company marketing segment? 34
- 35. Most of the time, we are blind to our users We guess, accept flawed info, and design for what WE want We’re not very good at guessing and intuition 35
- 36. Learn things about users to discover user insights 36 Option 2:
- 37. How? 37
- 38. The whole product team should have direct contact with real customers and real users in the real world doing their real tasks 38
- 39. Investigate the truth of users’ experiences 39
- 40. Use research techniques to collect quantitative and qualitative data 40
- 41. Over 100 techniques Interviews, focus groups, whiteboard sessions, surveys and questionnaires, contextual inquiry, ethnography, usability, attribution, participatory design, etc. 41
- 42. Users’ truths are multi-dimensional Measuring techniques are limited in scope 42
- 43. You need to take different, and appropriate, slices of the truth to really know it 43
- 44. You have to pick the right ones for your goal 44
- 45. AND these techniques require training and experience to avoid serious biases 45
- 46. It isn’t simply “talk with customers” 46
- 48. What we say can bias results Selling Effect How you say things can bias results “That’s not it!” Effect Asking questions can bias results Number Bias Demand Characteristics 48
- 49. Training and experience will reduce biases Some techniques themselves reduce biases by limiting the opportunity for biases 49
- 50. Which? 50
- 51. If done appropriately, these four techniques are the least biased for the non-scientist 51
- 52. • Structured Observation • Inquiry for Clarity • Talk-aloud Protocol • Descriptive Storytelling 52
- 53. Least biased because they don’t require*: • Designing empirically valid studies • Structuring data collection • Data analysis • Designing of questions 53 * They can also be used with techniques that do require these
- 54. Today I’m only focusing on insights from watching users AND only using one of the four techniques 54
- 55. Talk-aloud Protocol (aka protocol analysis, …) applied in an ethnographic setting 55
- 56. 56 Could apply in lots of techniques Why ethnographic? The Continuum Guessing Survey Interview Real context TruthLab
- 57. Facilitator Note-taker Observer 1. In-person live 2. Remote live 3. Video viewer 57
- 58. Talk-aloud Protocol Getting people to share thoughts aloud as they are experiencing things 58
- 59. 1. Description 2. Demo with a volunteer 3. Showpiece with volunteers 4. Practice in groups 5. Discuss 59
- 60. ➢ Description 60
- 61. When people do things, there is a rich experience going on inside them 61
- 62. Talk-aloud protocol is a way to understand some of that rich internal experience 62
- 63. We want to get “into their heads”, hear a projection of their experiences, and make the covert overt 63
- 64. We want to understand: • How did they do what the do • Why they do what they do • How they feel about it • Motivations, observations, frustrations, fears, awareness, etc. 64
- 65. This is not an interview You don’t ask them questions You just get them to talk as they do things 65
- 66. The Introduction (put at ease) • Thank you • Who is observing them and why • Not testing them • Permission to video • “Bumpers” • Questions about study – answers • Questions about product – no answer • Context – their real activity 66
- 67. The Training • Talk aloud • Whatever comes to mind • Continually 67
- 68. The Training • Thoughts • Feelings • Behaviors • What you are about to do, and why • What you expect to happen • What happened (result) 68
- 69. The Training • Perceptions • Opinions and preferences • Questions 69
- 70. Your Job: Keep them talking • Talkers and non-talkers • People tend to trail off • Weird to narrate your life 70
- 71. Prompts to keep them talking • “Please share what you’re thinking.” • “Please keep talking.” • “Tell us about what happened.” 71
- 72. What can go wrong? • Wandering • Censoring • Failure to speak • Pontificating & speaking for others • Designing • Temptation to talk/react 72
- 73. Wandering • People may wander • “My cousin is a doctor and…” • “After this, I have shopping…” • It’s normal • Steer them back politely 73
- 74. Censoring • Censoring is a signal • What are they not saying? • Watch faces, gestures, & posture • Reluctance to be negative • Personal and cultural 74
- 75. Censoring • Your opportunity to encourage • “Please feel free to tell us more about that.” • “You mentioned it was complicated. Please tell us more about that.” 75
- 76. Failure to speak • Sometimes prompting fails • Don’t push too far • Politely thank them and end the session 76
- 77. Pontificating and speaking for others • Want authentic experiences • Not abstracted detachment • “Well, anyone would expect …” • “Most people won’t like …” 77
- 78. Pontificating and speaking for others • Tough one to correct, but vital • “Please tell us how YOU feel.” • Try modeling the speech • “Please use ‘I’ statements like: • ‘I expected it to be …’ or • ‘I didn’t like …’” 78
- 79. Designing • Some people think you’re asking them to re-design • “This should be purple, and bigger, and over here, and…” • This is not a design opportunity • Steer them back to reporting on their experience 79
- 80. Temptation to talk / react • It will be very hard for you not to talk, ask questions, laugh, etc. • Don’t! • Think before you prompt • Remain interested but impartial 80
- 81. ➢ Demo with 1 volunteer 81
- 82. Tasks to choose from: • Planning a vacation to Florida • Research to buy a new car 82
- 83. 83 Cheat Sheet Introduction • Thank you • Who & why • Not testing you • Video permission • “Bumpers” • Questions • Study • Product • Context • Real Training • Talk aloud • What comes to mind • Continually • Thoughts • Feelings • Behaviors • Perceptions • Opinions • Preferences • Questions Your Job • Keep them talking • Prompt them • Wandering • Censoring • Failure to speak • Pontificating • Designing • Temptation
- 84. ➢ Showpiece with 3 volunteers • Participant user • Facilitator • Note taker 84 I’ll be your coach.
- 85. Tasks to choose from • The facilitator’s product • Find a new city to live in • Find best value in a pogo stick 85
- 86. 86 Cheat Sheet Introduction • Thank you • Who & why • Not testing you • Video permission • “Bumpers” • Questions • Study • Product • Context • Real Training • Talk aloud • What comes to mind • Continually • Thoughts • Feelings • Behaviors • Perceptions • Opinions • Preferences • Questions Your Job • Keep them talking • Prompt them • Wandering • Censoring • Failure to speak • Pontificating • Designing • Temptation
- 87. ➢ Practice in groups of 3 • Participant user • Facilitator • Note taker 87 I’ll be your coach wandering table to table.
- 88. Tasks to choose from • The facilitator’s product • Find a new dentist • Compare different lawnmower features 88
- 89. 89 Cheat Sheet Introduction • Thank you • Who & why • Not testing you • Video permission • “Bumpers” • Questions • Study • Product • Context • Real Training • Talk aloud • What comes to mind • Continually • Thoughts • Feelings • Behaviors • Perceptions • Opinions • Preferences • Questions Your Job • Keep them talking • Prompt them • Wandering • Censoring • Failure to speak • Pontificating • Designing • Temptation
- 90. ➢ Discuss 90
- 91. How far did I bring you? 91 Modified from Jeanine Spence’s learning progress model 2017. Current State Awareness Literacy Competency MasteryExperience
- 92. Attitude: • Curiosity • Openness • Willingness to accept whatever you discover 92 Parting advice
- 93. 93 Darren Kall • darrenkall@specificclarity.com • +1 937-516-9503 • www.specificclarity.com • @darrenkall Thank you