SlideShare a Scribd company logo
Out of the
Lab and
Into the Wild
TAKING USER EXPERIENCE RESEARCH TO THE USER
IN THEIR ENVIRONMENT JAY ZALTZMAN
BUREAU WEST
ABBY LEAFE
NEW LEAFE RESEARCH
Jay Zaltzman jay@bureauwest.comAbby Leafe abby@newleaferesearch.com
Part 1
WHY GET OUT OF THE LAB?
What’s in your
queue?
Cable customers tend to recall
this stuff…
…but not this stuff
Why?
Fear of judgment is only part of it
It’s hard to remember off the top of your head
Multi-viewer households
There’s lots that lab research is
good at
Usability
Nomenclature
Taxonomy
Task completion
Eye tracking
https://www.flickr.com/photos/dumfstar/8553474140
But there’s more to user
experience than usability
Decision-
making:
in the lab
vs real life
https://www.flickr.com/photos/vaxzine/2278300537
Case Study
Objective: A company was
developing a new navigation
app and wanted the best user
experience possible.
Approach: During an initial interview, participants discussed their
current navigation app and how they use it. Then they were shown the
new app. Then the (brave) researcher and a client got into the
participant’s car with the participant and drove to several locations using
the new app.
Case Study
Objective: Identify unmet needs
and potential products for dog
owners around the world.
Approach: Phase 1: week-long mobile diary during which participants
recorded their dogs’ activities, answered questions, and uploaded
photos and videos. Phase 2: online bulletin board discussion to uncover
unmet needs. Information from the diary served as stimulus and
catalyst for ideas.
Case Study
Objective: To understand how a
beta version of a new set-top box
operating system was functioning in
the real world.
Approach: One-on-one in-home interviews, using a task-based
approach to have participants walk through likely tasks (set up recording,
find On Demand programming, etc.). Respondents also kept a diary the
week prior to the interview to note any problems or pain points.
Case Study
Objective: Valpak had just launched
a responsive version of their website
and wanted to understand how its
mobile coupons performed in the
real world.
Approach: In part 1, respondents participated in a traditional one-on-
one usability interview in a lab setting. In part 2, they had 1 week to use
a Valpak mobile coupon in a store of their choice, and then report back
on the experience with photos and video via a mobile research app.
Part 2
QUESTIONS TO ASK YOURSELF
Is this study right for
ethnography?
 Look for experiences that you can’t recreate in
the lab
 Mobile apps are ripe for ethnography
because they are often designed for
use on-the-go
Should I attend?
There are plenty of good
reasons to be present…
or not
Pros & cons of in-person
ethnography
* Get to see things first hand
* Easier rapport
* See the whole environment
* Time consuming
* Can cost more
* Potential for observer bias
Pros & cons of remote
ethnography
* Privacy can create intimacy
* Can reach larger samples
* Less time and money
* See only what they show
* Don’t see body language
* Need to herd the kittens
What else should I be asking?
Do I go it alone?
(Consider partnering -- especially the first time)
What will the deliverable look like?
(Allocate resources)
Part 3
BLUEPRINT FOR CONDUCTING THE RESEARCH
Start with your target
Who do you really want?
May be similar to lab research
Consider screening for articulation,
openness, etc.
Make your expectations clear
In preparation for this study, please plan to clean
out your car so that two passengers beside
yourself can sit in it. This means you may have to
remove child car seats or other items from the
front and back seats of your car.
Also, please email us photos of your driver’s
license, car registration and proof of insurance.
Trust, but verify
Have respondents take a picture of
something that confirms their
qualification and send it to you (cable
bill, prescription medicine, etc.)
Consider pre-interviewing them on the
phone so you don’t waste time
Remote research options
Mobile screen capture
Diary (video, photos, voice, text)
Webcam
Many great providers to choose from
https://commons.wikimedia.org/wiki/File:FP_Satellite_icon.svg
In-home research
considerations
Safety and security -
for both research team
and respondents
Respect their space
Video
Make your expectations clear
Consumers:
 Don’t clean your house
 Don’t make snacks
 You will be recorded
 Expect 2-4 people
Observers:
 Cell phones off
 Dress professionally but no
company logos
 Don’t educate
 Poker face
Writing the guide
In-person:
◦ Allow time for set-up and getting comfortable
Remote:
◦ Be explicit
◦ Give examples if possible
◦ Give staggered timelines
(i.e. Complete X by Y)
All:
◦ Make it fun!
Keeping them engaged
Positive reinforcement – both privately and publicly
Question of the Day – so diaries don’t get repetitive
Bonus payments for certain activities
Other logistics
Have a dedicated videographer
Two hours and 3 observers is comfortable
What kind of release do you need?
Wear good socks
Don’t underestimate
the amount of data
you’ll get
Thoughts on reporting
Consider using a Livescribe pen
Think visually
Include photos and video when possible
Out of the Lab and Into the Wild! Mobile Ethnography for Richer UX Insights - Abby Leafe and Jay Zaltzman
Jay Zaltzman
jay@bureauwest.com
Abby Leafe
abby@newleaferesearch.com

More Related Content

PDF
Remote Usability Testing
PPT
Glass in the Classroom presentation - EDUCAUSE #eli2014 - 2/5/2014
PPTX
UX Testing in the library - SCLA 2016
KEY
Intro to UX for Programmers
PPTX
Look Beyond Data Trends - A Technique to Find Hidden Design Implications from...
PPTX
New Ideas for Designing Mobile Web Applications
PDF
Gamificatiom in E-Commerce (Xiao Chen)
PDF
Aggressive Research Wins C-Suite Buy-in (Michael Dutton & Colby Raley)
Remote Usability Testing
Glass in the Classroom presentation - EDUCAUSE #eli2014 - 2/5/2014
UX Testing in the library - SCLA 2016
Intro to UX for Programmers
Look Beyond Data Trends - A Technique to Find Hidden Design Implications from...
New Ideas for Designing Mobile Web Applications
Gamificatiom in E-Commerce (Xiao Chen)
Aggressive Research Wins C-Suite Buy-in (Michael Dutton & Colby Raley)

Viewers also liked (20)

PDF
Coping With the Stress of a Career in UX - Meredith Noble
PDF
Hollywood has a lot to answer for uxpa2015
PPTX
Development of the Code of Conduct (Richard Bellaver)
PPTX
Usability Best Practices for Adopting Responsive Design
PDF
The Mindfulness of Accessibility and Design Wu (Elle Waters)
PDF
Career Crossroads: Navigating Your Next Professional Move
PPTX
Future of UX in 10 Years (Kamaria Campbell)
PPTX
Breaking out of the Lab: UX Ethics in the Wild (Kara DeFrias)
PPTX
Mixed Methods Research in the Age of Big Data: A Primer for UX Researchers
PDF
Social ROI can Motivate Management to Support UX (Stephanie Rosenbaum)
PPTX
The Ubiquitous Eye Tracker (Joakim Isaksson, Tobii)
PPTX
IGNITE: How to Accidentally Create a Viral UX Infographic - Jennifer Aldrich
PDF
Goodbye, Focus Groups. Hello, Friendship Groups! New Techniques for Design Re...
PDF
UX In a Distributed Agile Environment: Making It Work - Krys Blackwood and Le...
PPTX
Designing Great Services Is Our Future: How Space, Time and Perspective Empow...
PDF
Win User Loyalty by Targeting Logic AND Emotion (Mike Donahue)
PDF
How UX Can Drive the Vision of Future Products - Arttu Niskasaari
PPTX
UXPA 2015 - IGNITE Introduction - Tony Fernandes
PDF
What the *UX?!? A Structured Approach to Tackling UX Strategy - Steve Dennin...
Coping With the Stress of a Career in UX - Meredith Noble
Hollywood has a lot to answer for uxpa2015
Development of the Code of Conduct (Richard Bellaver)
Usability Best Practices for Adopting Responsive Design
The Mindfulness of Accessibility and Design Wu (Elle Waters)
Career Crossroads: Navigating Your Next Professional Move
Future of UX in 10 Years (Kamaria Campbell)
Breaking out of the Lab: UX Ethics in the Wild (Kara DeFrias)
Mixed Methods Research in the Age of Big Data: A Primer for UX Researchers
Social ROI can Motivate Management to Support UX (Stephanie Rosenbaum)
The Ubiquitous Eye Tracker (Joakim Isaksson, Tobii)
IGNITE: How to Accidentally Create a Viral UX Infographic - Jennifer Aldrich
Goodbye, Focus Groups. Hello, Friendship Groups! New Techniques for Design Re...
UX In a Distributed Agile Environment: Making It Work - Krys Blackwood and Le...
Designing Great Services Is Our Future: How Space, Time and Perspective Empow...
Win User Loyalty by Targeting Logic AND Emotion (Mike Donahue)
How UX Can Drive the Vision of Future Products - Arttu Niskasaari
UXPA 2015 - IGNITE Introduction - Tony Fernandes
What the *UX?!? A Structured Approach to Tackling UX Strategy - Steve Dennin...
Ad

Similar to Out of the Lab and Into the Wild! Mobile Ethnography for Richer UX Insights - Abby Leafe and Jay Zaltzman (20)

PPTX
Don't Fear the User
PDF
Benchmarking Usability Performance
PDF
Getting Started with UX Research OCUX Camp CRossi Aug 2017
PPTX
The "Rocket Science" of Corporate Usability Testing - Boris Rudnik (CyberArk)
PPTX
Lab Report Template
PDF
Lean Ethnography
PDF
How to perform a user test?
PPTX
Remote Fieldwork: How observational studies elevated usability at AutoTrader.com
PPT
How to do usability testing and eye tracking
PDF
User Testing: Adapt to Fit Your Needs
PDF
UX Field Research Toolkit - A Workshop at Big Design - 2017
PDF
UX Field Research Toolkit - Updated for Big Design 2018
PPT
Homespun UX: Going Beyond Web Analytics
PPTX
Launch With Confidence! Integrate UX Research Throughout Development
PDF
Usability Testing On A Digital Product
PPTX
Planning & Conducting a Research Study
PDF
Analytic emperical Mehods
PDF
Planning, Conducting, and Analyzing User Research
PDF
DrupalCon Austin: UX Bootcamp workshop
PPTX
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
Don't Fear the User
Benchmarking Usability Performance
Getting Started with UX Research OCUX Camp CRossi Aug 2017
The "Rocket Science" of Corporate Usability Testing - Boris Rudnik (CyberArk)
Lab Report Template
Lean Ethnography
How to perform a user test?
Remote Fieldwork: How observational studies elevated usability at AutoTrader.com
How to do usability testing and eye tracking
User Testing: Adapt to Fit Your Needs
UX Field Research Toolkit - A Workshop at Big Design - 2017
UX Field Research Toolkit - Updated for Big Design 2018
Homespun UX: Going Beyond Web Analytics
Launch With Confidence! Integrate UX Research Throughout Development
Usability Testing On A Digital Product
Planning & Conducting a Research Study
Analytic emperical Mehods
Planning, Conducting, and Analyzing User Research
DrupalCon Austin: UX Bootcamp workshop
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.
Ad

More from UXPA International (20)

PPTX
UXPA 2023: Start Strong - Lessons learned from associate programs to platform...
PDF
UXPA 2023: Disrupting Inaccessibility: Applying A11Y-Focused Discovery & Idea...
PDF
UXPA 2023 Poster: ESG & Sustainable UX
PDF
UXPA 2023 Poster: The Two Tracks of UX Under Agile: Tactical and Strategic
PDF
UXPA 2023: Data science and UX: Smarter together
PDF
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
PDF
UXPA 2023 Poster: Are virtual spaces the future of video conferencing?
PDF
UXPA 2023: Learn how to get over personas by swiping right on user roles
PDF
UXPA 2023: F@#$ User Personas
PDF
UXPA 2023 Poster: Pocket Research Guide - Empower your Solution and Foster Cu...
PDF
UXPA 2023: Experience Maps - A designer's framework for working in Agile team...
PDF
UXPA 2023 Poster: Atomic Research in Practice: Using a Feedback Repository to...
PDF
UXPA 2023 Poster: Leveraging Dial Testing To Measure Real-Time User Frustrati...
PDF
UXPA 2023: UX Enterprise Story: How to apply a UX process to a company withou...
PPTX
UXPA 2023: High-Fives over Zoom: Creating a Remote-First Creative Team
PPTX
UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & ...
PDF
UXPA 2023 Poster: Improving the Internal and External User Experience of a Fe...
PDF
UXPA 2023 Poster: 5 Key Findings from Moderated Accessibility Testing with Sc...
PDF
UXPA 2023: Lessons for new managers
PDF
UXPA 2023: Redesigning An Automotive Feature from Gasoline to Electric Vehicl...
UXPA 2023: Start Strong - Lessons learned from associate programs to platform...
UXPA 2023: Disrupting Inaccessibility: Applying A11Y-Focused Discovery & Idea...
UXPA 2023 Poster: ESG & Sustainable UX
UXPA 2023 Poster: The Two Tracks of UX Under Agile: Tactical and Strategic
UXPA 2023: Data science and UX: Smarter together
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
UXPA 2023 Poster: Are virtual spaces the future of video conferencing?
UXPA 2023: Learn how to get over personas by swiping right on user roles
UXPA 2023: F@#$ User Personas
UXPA 2023 Poster: Pocket Research Guide - Empower your Solution and Foster Cu...
UXPA 2023: Experience Maps - A designer's framework for working in Agile team...
UXPA 2023 Poster: Atomic Research in Practice: Using a Feedback Repository to...
UXPA 2023 Poster: Leveraging Dial Testing To Measure Real-Time User Frustrati...
UXPA 2023: UX Enterprise Story: How to apply a UX process to a company withou...
UXPA 2023: High-Fives over Zoom: Creating a Remote-First Creative Team
UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & ...
UXPA 2023 Poster: Improving the Internal and External User Experience of a Fe...
UXPA 2023 Poster: 5 Key Findings from Moderated Accessibility Testing with Sc...
UXPA 2023: Lessons for new managers
UXPA 2023: Redesigning An Automotive Feature from Gasoline to Electric Vehicl...

Recently uploaded (20)

PDF
SEVA- Fashion designing-Presentation.pdf
PDF
Urban Design Final Project-Context
PPTX
AD Bungalow Case studies Sem 2.pptxvwewev
PDF
Test slideshare presentation for blog post
PDF
BRANDBOOK-Presidential Award Scheme-Kenya-2023
PDF
Key Trends in Website Development 2025 | B3AITS - Bow & 3 Arrows IT Solutions
PPT
Machine printing techniques and plangi dyeing
PDF
GREEN BUILDING MATERIALS FOR SUISTAINABLE ARCHITECTURE AND BUILDING STUDY
PDF
Design Thinking - Module 1 - Introduction To Design Thinking - Dr. Rohan Dasg...
PDF
intro_to_rust.pptx_123456789012446789.pdf
PPTX
Implications Existing phase plan and its feasibility.pptx
PDF
YOW2022-BNE-MinimalViableArchitecture.pdf
PPTX
YV PROFILE PROJECTS PROFILE PRES. DESIGN
PPTX
Causes of Flooding by Slidesgo sdnl;asnjdl;asj.pptx
PDF
Integrated-2D-and-3D-Animation-Bridging-Dimensions-for-Impactful-Storytelling...
PPTX
Entrepreneur intro, origin, process, method
PPTX
DOC-20250430-WA0014._20250714_235747_0000.pptx
PDF
Chalkpiece Annual Report from 2019 To 2025
PDF
UNIT 1 Introduction fnfbbfhfhfbdhdbdto Java.pptx.pdf
PDF
Introduction-to-World-Schools-format-guide.pdf
SEVA- Fashion designing-Presentation.pdf
Urban Design Final Project-Context
AD Bungalow Case studies Sem 2.pptxvwewev
Test slideshare presentation for blog post
BRANDBOOK-Presidential Award Scheme-Kenya-2023
Key Trends in Website Development 2025 | B3AITS - Bow & 3 Arrows IT Solutions
Machine printing techniques and plangi dyeing
GREEN BUILDING MATERIALS FOR SUISTAINABLE ARCHITECTURE AND BUILDING STUDY
Design Thinking - Module 1 - Introduction To Design Thinking - Dr. Rohan Dasg...
intro_to_rust.pptx_123456789012446789.pdf
Implications Existing phase plan and its feasibility.pptx
YOW2022-BNE-MinimalViableArchitecture.pdf
YV PROFILE PROJECTS PROFILE PRES. DESIGN
Causes of Flooding by Slidesgo sdnl;asnjdl;asj.pptx
Integrated-2D-and-3D-Animation-Bridging-Dimensions-for-Impactful-Storytelling...
Entrepreneur intro, origin, process, method
DOC-20250430-WA0014._20250714_235747_0000.pptx
Chalkpiece Annual Report from 2019 To 2025
UNIT 1 Introduction fnfbbfhfhfbdhdbdto Java.pptx.pdf
Introduction-to-World-Schools-format-guide.pdf

Out of the Lab and Into the Wild! Mobile Ethnography for Richer UX Insights - Abby Leafe and Jay Zaltzman

  • 1. Out of the Lab and Into the Wild TAKING USER EXPERIENCE RESEARCH TO THE USER IN THEIR ENVIRONMENT JAY ZALTZMAN BUREAU WEST ABBY LEAFE NEW LEAFE RESEARCH Jay Zaltzman jay@bureauwest.comAbby Leafe abby@newleaferesearch.com
  • 2. Part 1 WHY GET OUT OF THE LAB?
  • 4. Cable customers tend to recall this stuff…
  • 6. Why? Fear of judgment is only part of it It’s hard to remember off the top of your head Multi-viewer households
  • 7. There’s lots that lab research is good at Usability Nomenclature Taxonomy Task completion Eye tracking https://www.flickr.com/photos/dumfstar/8553474140
  • 8. But there’s more to user experience than usability
  • 9. Decision- making: in the lab vs real life https://www.flickr.com/photos/vaxzine/2278300537
  • 10. Case Study Objective: A company was developing a new navigation app and wanted the best user experience possible. Approach: During an initial interview, participants discussed their current navigation app and how they use it. Then they were shown the new app. Then the (brave) researcher and a client got into the participant’s car with the participant and drove to several locations using the new app.
  • 11. Case Study Objective: Identify unmet needs and potential products for dog owners around the world. Approach: Phase 1: week-long mobile diary during which participants recorded their dogs’ activities, answered questions, and uploaded photos and videos. Phase 2: online bulletin board discussion to uncover unmet needs. Information from the diary served as stimulus and catalyst for ideas.
  • 12. Case Study Objective: To understand how a beta version of a new set-top box operating system was functioning in the real world. Approach: One-on-one in-home interviews, using a task-based approach to have participants walk through likely tasks (set up recording, find On Demand programming, etc.). Respondents also kept a diary the week prior to the interview to note any problems or pain points.
  • 13. Case Study Objective: Valpak had just launched a responsive version of their website and wanted to understand how its mobile coupons performed in the real world. Approach: In part 1, respondents participated in a traditional one-on- one usability interview in a lab setting. In part 2, they had 1 week to use a Valpak mobile coupon in a store of their choice, and then report back on the experience with photos and video via a mobile research app.
  • 14. Part 2 QUESTIONS TO ASK YOURSELF
  • 15. Is this study right for ethnography?  Look for experiences that you can’t recreate in the lab  Mobile apps are ripe for ethnography because they are often designed for use on-the-go
  • 16. Should I attend? There are plenty of good reasons to be present… or not
  • 17. Pros & cons of in-person ethnography * Get to see things first hand * Easier rapport * See the whole environment * Time consuming * Can cost more * Potential for observer bias
  • 18. Pros & cons of remote ethnography * Privacy can create intimacy * Can reach larger samples * Less time and money * See only what they show * Don’t see body language * Need to herd the kittens
  • 19. What else should I be asking? Do I go it alone? (Consider partnering -- especially the first time) What will the deliverable look like? (Allocate resources)
  • 20. Part 3 BLUEPRINT FOR CONDUCTING THE RESEARCH
  • 21. Start with your target Who do you really want? May be similar to lab research Consider screening for articulation, openness, etc.
  • 22. Make your expectations clear In preparation for this study, please plan to clean out your car so that two passengers beside yourself can sit in it. This means you may have to remove child car seats or other items from the front and back seats of your car. Also, please email us photos of your driver’s license, car registration and proof of insurance.
  • 23. Trust, but verify Have respondents take a picture of something that confirms their qualification and send it to you (cable bill, prescription medicine, etc.) Consider pre-interviewing them on the phone so you don’t waste time
  • 24. Remote research options Mobile screen capture Diary (video, photos, voice, text) Webcam Many great providers to choose from https://commons.wikimedia.org/wiki/File:FP_Satellite_icon.svg
  • 25. In-home research considerations Safety and security - for both research team and respondents Respect their space Video
  • 26. Make your expectations clear Consumers:  Don’t clean your house  Don’t make snacks  You will be recorded  Expect 2-4 people Observers:  Cell phones off  Dress professionally but no company logos  Don’t educate  Poker face
  • 27. Writing the guide In-person: ◦ Allow time for set-up and getting comfortable Remote: ◦ Be explicit ◦ Give examples if possible ◦ Give staggered timelines (i.e. Complete X by Y) All: ◦ Make it fun!
  • 28. Keeping them engaged Positive reinforcement – both privately and publicly Question of the Day – so diaries don’t get repetitive Bonus payments for certain activities
  • 29. Other logistics Have a dedicated videographer Two hours and 3 observers is comfortable What kind of release do you need? Wear good socks
  • 30. Don’t underestimate the amount of data you’ll get
  • 31. Thoughts on reporting Consider using a Livescribe pen Think visually Include photos and video when possible