Remote User Testing - MSU 12 Nov2021
Download as PPTX, PDF0 likes346 views
The document provides an overview of remote user testing. It discusses why remote testing should be used, as well as when it should not be used. It then covers different types of remote testing, popular tools for remote testing, tips for effective remote testing, and answers questions from participants. The document concludes by discussing analysis and deliverables from remote testing projects.
Remote User Testing - MSU 12 Nov2021
- 1. Remote User Testing Chris Farnum M S U X A P 2 W D a y o f P r o f e s s i o n a l i z a t i o n | N o v 1 2 , 2 0 2 1
- 2. Hello! O h w a i t , i t l o o k s l i k e w e ’ r e d o i n g a r e m o t e u s e r t e s t … Part 1 – Presentation: An intro to remote user testing and research • Why you should • When you shouldn’t • Types of testing • Tools • Tips • Q&A Part 2 – Activity: Optimal Workshop @crfarnum
- 3. @crfarnum Q - How many of you have done remote research?
- 4. @crfarnum Introductions C h r i s F a r n u m , L e a d U s e r E x p e r i e n c e R e s e a r c h e r – G e n e r a l M o t o r s 20+ years as an Information Architect / UX Designer 3+ years as a User Experience Researcher Embedded in a UX design team focused on mobile apps and web apps. Librarian in a former career
- 5. @crfarnum Argus Associates ProQuest Compuware West Bloomfield Township Public Library Enlighten Blue Cross Blue Shield of Michigan General Motors x2
- 6. @crfarnum Why Remote? COVID 19 Lab-less Logistics / cost Recruiting without geo limits Scheduling Flexibility
- 8. @crfarnum When NOT to use remote testing? Face to face Observe non-digital interactions Contextual research Need precise measurements and/or recording Proprietary or secret content
- 9. @crfarnum Photo by Alejandro Escamilla on Unsplash. More about contextual research via UX Collective When physical setups, camera angles matter... https://www.lextant.com/remote-research/
- 10. @crfarnum Tools Testing Platforms • Conferencing software + screenshare • UserTesting.com • UserZoom • Usability Hub • Qualtrics • DScout Prototype / Stimulus • AxShare • Proto.io • InVision • Optimal Workshop…
- 11. @crfarnum
- 12. @crfarnum Types of Remote Testing • Prototype testing • First click • Card Sorting (open / closed) • Tree / Nav testing • A/B testing • Competitive analysis • System Usability Scale – benchmarking • Visual aesthetic ratings • New concept / design thinking testing • User needs research • Diary contextual studies
- 13. @crfarnum Chalkmark – Optimal Workshop
- 14. @crfarnum Moderated v Unmoderated Moderated • live/real-time testing • like a Zoom call Unmoderated • Set up tasks/questions and launch • Participants complete on their own
- 15. Moderated - Flexible - Ask follow-up questions - Ability to explain stimulus / proto - Generates more empathy - Team may be more engaged - … @crfarnum Unmoderated - Ability to ask targeted questions - Consistent questions / less variables - Less time needed to run sessions - Efficient for A/B, card sort, surveys, etc. - Easier to get quant-esque, e.g. SUS - …. Hybrid? Q – Have you tried both? What do you prefer?
- 16. @crfarnum Recruiting Yes! It matters who you test with. AND it depends on… • Amount of background context, experience needed • Usability or desirability? • Time / budget / tools available
- 17. On a budget - Tear off flyer or Social media / forum post - Email address or screener survey link - Schedule with online calendar, Doodle @crfarnum All-inclusive - Vendor panel - Custom screen questions - Site intercept - Set schedule availability https://templates.office.com/en-us/flyer-with-tear-off-tabs-and-art-tm16392565 https://www.userzoom.com/participant-recruiting/what-are-screeners-and-why-are-they-important/
- 18. @crfarnum Testing for Mobile Designs Can be a challenge… • MANY device screen sizes • Extra work to build proto • Can you share a URL? (info security) • Host solution viewer code It’s worth it when… • Touch interactions matter (i.e., side-scroll) • Testing new patterns, navigation • Testing live apps
- 19. Testing Mobile on Mobile - Participant screen shares from own device - Hosted proto - Live app, downloaded? Tip: make sure to test proto while building it to prevent rework and surprises. @crfarnum Testing Mobile on Desktop - Participant views on desktop - Web based proto viewer displays screen frame - Screen cast from moderator device, share control https://www.flaticon.com/free-icon/responsive_896405?term=mobile%20devices&related_id=896405 Q – Have you ever tried testing a mobile design?
- 20. @crfarnum UXRs and the Team If you are a UXR… • Be part of the process – Agile or other • Communicate that you can support both user research and testing • Get involved with designers and stakeholders early • Set goals • Understand audience • Plan scenarios and protos • Ask them to observe, take notes, ask questions • Be visible – present your work, attend critiques
- 21. Moderated ask t eam t o observe in real t ime @crfarnum https://www.usertesting.com/blog/invite-observers-live-conversation Unmoderated share videos, creat e highlight reels
- 22. @crfarnum What should designers know about test prep? If you are a Designer working with a UXR… • Build protos that demonstrate just-enough • Key use cases (usually not edge cases) • Need to be “tight” so that participants can navigate without a moderator • Consistent details matter • Collaborate – build the script and proto in tandem • Mobile design? Test it on your own device(s).
- 23. design like you’re right and test like you’re wrong @crfarnum Expect explosions leave t ime and space f or it erat ions Images via https://www.flaticon.com/ Jeff Patton OnAgile2017 Q&A via Agile Alliance Q – Have you ever been asked to “validate” a design?
- 24. @crfarnum What makes a good test plan/script? Ingredients for a successful remote test: • Make sure to ask participants to “think aloud” and to turn on their video camera • Give participants a scenario, goal, situation, and even an identity for task-based testing • Use visuals to explain complex background and scenarios • Likert scales for ratings, multiple choice – great for generating comments! • Ask clear, straightforward questions – especially for unmoderated scripts
- 25. T h i s i s M o r r i s F a m i l y. F r a n k a n d D e n i s e a r e t h e p a r e n t s , a n d S a m a n d D i a n n a a r e t h e i r t e e n a g e k i d s . F r a n k , a n O n S t a r m e m b e r w i t h a p a i d p l a n , r e c e n t l y l e a r n e d a b o u t i t s m o b i l e a p p a n d d e c i d e d t o c h e c k i t o u t f r o m t h e a p p s t o r e … @crfarnum Prototype credit – Claire Koo S u p p o s e D i a n n a i s g o i n g t o t h e m a l l w i t h h e r f r i e n d s . H e r f r i e n d c a m e t o p i c k h e r u p . O n t h e i r w a y t o t h e m a l l , a c a r v e e r e d s u d d e n l y a n d c o l l i d e d w i t h t h e m …
- 26. @crfarnum Analysis and Deliverables Typical Artifacts • Test plan • Script • Excel analysis • Debrief session notes • Report – often slides • Executive summary • Video highlight reels
- 27. Reporting Tips for Remote Testing - Use test platform analysis tools to accelerate analysis and reporting - If your study is qualitative, make that obvious - Use video clips when it makes sense @crfarnum 86.9% 5 - Very Easy 10.1% 4 – Easy 3% 3 - Neutral Bar chart via UserTesting.com UserTesting.com auto-creates metrics that emphasize the number of participants .
- 28. @crfarnum Challenges & Frustrations • Prototype glitches and incompatibilities • Hosting for video files • Bad egg participants (especially unmoderated) • Account limits • Reports ;-)
- 29. @crfarnum Triumphs • Remote testing resulted in 2x-3x more studies • Able to reach narrow segments – EV drivers • Well suited to pace of sprints as design group transitioned to Agile • Concept testing for new products and a large-scale redesign
- 30. @crfarnum Likely to Recommend? 5 It’s a game-changer! Start small, take advantage of trials and existing solutions and tools.
- 31. @crfarnum Your Questions? Thanks for listening! Chris Farnum crfarnum@gmail.com Christopher.Farnum@gm.com @crfarnum
- 32. @crfarnum Break / Coffee Time! – 5 mins
- 34. @crfarnum Imagine you have a research project f or Technology A t MSU … https://tech.msu.edu/
- 35. @crfarnum Background ( 1 0 0 % F I C T I O N A L – C h r i s m a d e t h i s u p . ) https://tech.msu.edu/ MSU IT wants to evaluate the tech.msu.edu to plan priorities for future enhancements. You’ve done stakeholder interviews and have learned: • Key target audiences: students, faculty, staff, visitors. In the last redesign, they organized the site/home page according to these audiences. Now they want to know if this structure is working well. • Call volume for the help desk is very high at move-in times and during peak weeks during the semester. IT would like students and faculty to use more self-help to reduce call wait times. • Support for distance learning and remote group projects is a continuing challenge. The department is rolling out Microsoft Teams for class group projects. They want to get more people on campus to adopt it. They have started training session events, but sign ups are slow. • Analytics show that tip sheets and documentation are nearly always accessed via external (Google) search or site search rather than browsing traffic from the Tech site.
- 36. @crfarnum Imagine that you have Optimal Workshop available as a toolkit for this project… https://www.optimalworkshop.com/ Card Sort (open/closed/hybrid) Tree Test First Click Survey
- 37. @crfarnum Instructions – 30 min exercise https://tech.msu.edu/ Step 1: In your group discuss and decide the following (put your answers on a shared slide or doc) • Take 5 mins to review the site. • Decide your test goals, audience, and the type of insights that you need to help move the project forward. • Stuck? Try filling in the blanks: In order to know ___<main test goals>___ we need to learn __<missing insights/knowledge gap>___ by getting input from __target audience___. • Example: In order to design the best way to teach people how to start an e-scooter we need to learn what’s difficult about it by getting input from people in an urban environment that haven’t tried an e-scooter yet. Step 2: Choose a test method from Optimal Workshop. • Explain why you chose this method. • Will this study be moderated or unmoderated? • What questions/stimuli/assets will you need for your test? • (If time allows) Try getting started with a draft of your test. Bonus question – how will you recruit? Step 3: We’ll regroup and discuss https://www.optimalworkshop.com/
- 38. @crfarnum Discussion / Wrap up Go Green! What was the goal of your study? What method did you choose? Why? What was difficult or easy about the activity? What did you learn? Thanks again for inviting me!