Flotree requirements interview mistakes
2 likes450 views
User requirements interviews often go wrong when designers fail to properly understand user work contexts. Mistakes include not observing users in the field, accepting non-user representatives, and not getting low-level work details. To get useful data, designers must see live or retrospective work, use "magic words" to elicit specifics, and assume a partnership role rather than acting as interrogators. Even one good field interview provides better insights than none.
Flotree requirements interview mistakes
- 1. ProductCamp Vancouver 2011 User Requirements Interviewing Mistakes Recognizing and preventing them Dave Flotree Program Manager dave.flotree@incontextdesign.com www.incontextdesign.com 206-546-3833 Copyright © 2011 InContext Enterprises Inc. All rights reserved.
- 2. InContext Design Developers of Contextual Design A repeatable end-to-end design process Used to develop business systems, products web-based applications, consumer goods… Used and taught in universities and companies all over the world Supporting any methodology: RUP, Agile, Corporate … Principals Karen Holtzblatt, co-founder and CEO • Recognized worldwide as an expert on user-centered design Hugh Beyer, co-founder and CTO • The technology backbone to all our designs and processes InContext Founded – 1992 >20 professionals Offices in Boston and Chicago
- 3. Solving the problem of design Design the optimal match: Products and systems embody work practice Design must support and extend user intent The User’s Work Model has: Language Work Structure Work Flow Intention Result The Product/System’s Work Model has: Language System Structure Features Concepts
- 4. Users have all the knowledge…. …but they can’t tell us in a way that reliably informs product design Users do the work, they don’t self-analyze how they do the work The more expert they are, the more invisible the work details are to them To find the work practice, we need to be with them while they work Or re-create recently done work Field interviews done as contextual inquiries reliably: Uncover what customers do and need See the underlying details And as important as the ―whats‖, discover the ―whys‖
- 5. Principles to guide contextual inquiry interviews Context Understand user needs in the context of their work Go to where people work or live and talk to them while they work Partnership The user is the expert and has all the knowledge Work with customers as partners in inquiry Use the model of the master-apprentice Interpretation It’s not the facts that matter, it’s the interpretation of the facts Uncover the meaning and implications of user action and language Focus Listen and probe from a clear intention Acknowledge your entering assumptions, and then assume you are wrong
- 6. Mistake 1: Pretending you are in the field These places are not the field: A focus group The usability lab A user conference A conference room in the workplace Even the office or home may not be the field • If the work is done elsewhere
- 7. What to do to avoid the mistake Expect that users won’t understand right away Be ready to politely move the user into the work space It’s actually rude not to move the user • Otherwise you are wasting their time Travel to the field Even when you have little time or budget • It’s where the customer data is, so help your management understand this – Even a few field interviews can pick up requirements that might otherwise be missed • Look for local people who do the work to save time and money Remote meeting tools are your fallback, but • Start with in-person interviews first – And then use a collaboration tool if the work is dominated by web or tool interaction • Be sure the user is in the real workplace, doing real work
- 8. Mistake 2: Accepting a representative user These people are not who you want to interview Someone who used to do the job • Subject matter experts, for example Someone who tells others how to do the job but doesn’t do it • Managers People in your company who do the same job as the customers • Unless you are designing an internal system What to do to avoid the mistake Don’t assume that you have to interview the non-users Let them be heard in other ways • Stakeholder interviews help set project focus • Do ―courtesy‖ interviews – Short, non-contextual interviews that you do because you have to • Just be careful to separate this data from the field data • Let them be on the team and gather the real data too
- 9. Mistake 3: The “I can’t see live work” excuse Sometimes the work you need to see: Is intermittent or infrequent Takes place over long periods of time Is highly sensitive or confidential So you assume that you can’t see the work What to do to avoid the mistake Conduct a retrospective account interview • Re-create the actual work done in the past • Within the last two weeks is best • Re-created with actual work artifacts Be careful with retrospective accounts • You only care about the current and past – Don’t ask the user to make up the future • The user will tend to skip over parts of the story – It’s your job to recognize skips and help the user to fill them in
- 10. Mistake 4: Not getting low level details Recognize an abstraction when you hear one ―Generally, typically, normally, our company’s policy…‖ Abstractions may tell us there’s a problem or opportunity • But they do not give us the detail we need for design What to do to avoid the mistake Don’t ask users what they typically, generally, or normally do — you are asking for a generalization Stick with a real instance and play it out live or retrospectively And use the ―magic words‖ from Mistake 5…
- 11. Mistake 5: Assuming users can tell you… ...if you just ask Don’t expect users to articulate what they do without your help The more expert the users… • …the less able they are to articulate what they do and why They do the work, not think about the work What to do to avoid the mistake The user is the expert — let them lead Your job is to help the user articulate their work practice, so use the ―magic words‖ • ―When was the last time you…?‖ • ―Can you show me what you did?‖
- 12. Mistake 6: Asking “Why?” or open-ended questions Interpretation is the data Customer Create a shared understanding of what is going on Inquire into the meaning of Fact customer action and words tune the interpretation What to do to avoid the mistake Hypothesis Offer interpretations, not just open-ended questions Implication Design Idea
- 13. Mistake 7: Not being honest about the reaction Sometimes a ―no‖ isn’t said as ―no‖ People want to be helpful and nice! What to do to avoid the mistake Listen for the ―no‖ • Huh? • Umm... could be • ―They‖ would like it • ―Yes‖ comes with elaboration Watch for non-verbal cues
- 14. Mistake 8: Falling into ineffective interview styles The Traditional Interviewer ―I Ask/You Answer‖ The Court Reporter Writing down everything without finding underlying intents The Police Interrogator Grilling the customer The Agenda Follower Paying attention to your personal focus, not the user’s work The ―Polite‖ Interviewer Letting the user go on tangents that are not in your focus Thinking it is rude to be curious The Mad Inventor Constantly offering design ideas and solutions
- 15. What to do to avoid the mistake Knowing the ineffective relationship models creates awareness Use these styles as triggers to change your approach Apprenticeship is the preferred relationship model Listen, learn, be humble, don’t judge And assume that people do things for a reason Return to the ongoing work It always keeps you in the apprenticeship model
- 16. Mistake 9: Expecting the interview to run itself The interview is not casual or informal It requires intense focus on capturing a lot of detail in a limited time While following the master-apprentice relationship model How you run the interview is key—it’s a partnership The way you conduct the interview ―trains― the user • Users will follow your lead So make sure expectations are clear Users have no frame of reference for this kind of interview So it can take awhile to adapt Some users will adapt quickly, others take a long time • And then it will feel like hard work on your part for the entire interview • But that's o.k., that’s your job!
- 17. Mistake 10: Forgetting that one good field interview… …is much better than no interviews And two interviews is twice as good And so on…. Success is getting any user data into the project And then having a customer-centered process as part of standard development If using contextual data is new, small steps are often needed • It often involves organizational culture change
- 18. Get the most out of data after the interview Don’t write a trip report or debrief by yourself Use a process like an interpretation session with at least one other person Other perspectives reveal more insights Capture the data in a way that useful for organizing it, e.g. virtual Post-It notes Build an affinity diagram Reveals underlying pattern: intent, strategy, structure, and scope Shows what matters to the entire population, and keeps variations that matter Consider if other work models will be valuable Brainstorm (vision) product concepts / requirements Grounded in the customer data — no more designing in isolation Work out the details Interaction patterns, storyboarding, and system structure ensure a coherent experience Test with customers Mock up the design in rough paper prototypes Rough mockups test structure and coherence of the design
- 19. Contextual Design Requirements & Solutions Contextual Inquiry Talk to specific customers in the field What matters to Interpret the data as a team to capture key issues users – Interpretation Session characterizing what they do Work Models and Consolidate data across customers for a full market view Affinity Diagramming New ideas and Visioning Redesign people’s work with new technology ideas direction Define & Validate Concepts Storyboarding Work out the details of particular tasks and roles Redesign activities and technology to User Environment provide value Design system to support this new work Design Paper Mock-Up Mock up the interface using interaction patterns for testing Interviews Iterate the system with Interaction & Visual users Design and test the final look and user experience Design
- 20. Thank you! Dave Flotree dave.flotree@incontextdesign.com www.incontextdesign.com