Note-Taker's Perspective During Usability Testing (Kristen Davis & Dana Douglas)
Download as PPT, PDF2 likes629 views
The document discusses the importance of effective note-taking during usability testing. It notes that note-taking requires preparation and an understanding of the goals and metrics of the test. The document outlines exercises where participants watch video clips of usability tests and record notes based on predefined goals and scoring metrics. Lessons learned emphasize understanding the goals of the study, defining appropriate metrics, and planning for analysis before collecting data to improve note-taking.
Note-Taker's Perspective During Usability Testing (Kristen Davis & Dana Douglas)
- 1. The Note-Taker's Perspective During Usability Testing: Recognizing What's Impor tant, What's Not User Focus 2012 October 19, 2012 www.userworks.com (301) 431-0500 Kristen Davis & Dana Douglas
- 2. Note-Taking Misconceptions No preparation needed Take the notes, then figure out how to analyze and summarize the data Anyone can do it Attention to detail, fast typing are the key skills A one-size-fits-all approach will work for all practitioners and for all products under evaluation The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 2
- 3. Why is Ef fective Note-Taking Impor tant? To improve the data analysis process by: • Capturing the appropriate data the first time • Making analysis more efficient • Increasing accuracy • Identifying trends easily The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 3
- 4. Exercise 1: Taking Notes
- 5. Exercise Record your observations as you watch this video clip The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 5
- 6. How Did You Do? Did you capture the key findings? Did you miss anything? The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 6
- 7. Exercise 2: Know the Study
- 8. Practice Exercise Product: NIHSeniorHealth.gov website Participant: Seniors who look for health information online Task: Locate a specific piece of information Study Goals: • What navigation path do participants use when looking for specific pieces of information? • Do par ticipants notice and use the pagination? • Does the visual design treatment used in the left hand navigation clearly indicate the user’s location within a specific health topic? Task Goals: • Are par ticipants able to locate information on the 2 nd page? • Are the left hand navigation labels clear? The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 8
- 9. Exercise Record your observations while watching this video clip The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 9
- 10. How Did You Do? Did you capture the key findings? Did you miss anything? Did you note: • If the participant noticed and used the pagination? • If the participant was able to locate information on the 2nd page? The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 10
- 11. Lessons Learned: Understand the Goals Goals should be set prior to data collection in order to: • Record appropriate data • Target the type of data that will be useful when reporting the findings The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 11
- 12. Exercise 3: Know the Metrics
- 13. Practice Exercise Product: Blood glucose meter Participant: Person with diabetes who uses a meter Task: Change default settings in the meter Rating scale for scoring task completion: • Completed with ease: Participant easily completed the task • Completed with minor dif ficulty: Participant somewhat struggled to complete the task (e.g., attempted more than two paths) • Completed with major dif ficulty: Participant significantly struggled to complete the task (e.g., attempted numerous paths and/or used customer service) • Failed to complete: Participant was unable to complete the task The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 13
- 14. Exercise Record your observations while watching this video clip The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 14
- 15. How Did You Do? What score did you give the participant? • Completed with ease: Participant easily completed the task • Completed with minor dif ficulty: Participant somewhat struggled to complete the task (e.g., attempted more than two paths) • Completed with major dif ficulty: Participant significantly struggled to complete the task (e.g., attempted numerous paths and/or used customer service) • Failed to complete: Participant was unable to complete the task The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 15
- 16. Lessons Learned: Define Metrics Defining the task completion rating scale prior to data collection allows: • Data to become less subjective • Note-taker to record the scores in a systematic and consistent way Some additional metrics: • Clickpath • Time on task • Shorthand codes Example: Number of times a participant clicked the pagination The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 16
- 17. Summar y
- 18. Keys to Success Both moderator and note-taker need to know the product and the issues of interest Be thoroughly familiar with whatever tool you are using Plan for categorization and data analysis before data collection Don’t try to capture comments verbatim; interpret what is noteworthy as you go The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 18
- 19. Questions/Comments? Kristen Davis Dana Douglas UserWorks, Inc. UserWorks, Inc. User Experience Specialist/ User Experience Specialist Note-Taker Extraordinaire kdavis@userworks.com ddouglas@userworks.com www.userworks.com (301) 431-0500 The Note-Taker's Perspective During Usability Testing User Focus 2012 Kristen Davis and Dana Douglas- UserWorks, Inc. 19
Editor's Notes
- #2: Copyright UserWorks, Inc. 2008
- #3: Dana: Let’s begin with a few misconceptions about note-taking. Preparation = getting familiar with the product being tested and understanding project and task goals Copyright UserWorks, Inc. 2008
- #4: Dana: So why is important to be a good note taker? [Recording] So you don’t need to re-watch the recordings All of these help the report basically write itself. Copyright UserWorks, Inc. 2008
- #5: Kristen: Let’s start with the first exercise. Copyright UserWorks, Inc. 2008
- #6: Kristen: Let’s say you are being pulled into a study at the last minute. I’m going to show you a video clip and I want you to take notes on what you observe. I’m not going to give you any background to the study shown in the clip or to the situation. I just want you to write down what you think is important to note. [Video Clip: NIHSeniorHealth.gov phase 1B P4 13:57-16:59] Copyright UserWorks, Inc. 2008
- #7: Kristen: Without having the appropriate context it can be difficult to take meaningful notes. So did you capture the key findings? Did you miss anything? It’s probably safe to say with a little more context you could improve your notes and therefore the report. Copyright UserWorks, Inc. 2008
- #8: Kristen: So let’s try this activity again, this time with some background information on the study. Copyright UserWorks, Inc. 2008
- #9: Kristen: This is the same study, but a different information retrieval task… For this exercise let’s focus on the goals related to pagination. So either if the participant noticed and used the pagination and if he was able to successfully locate the information that he was asked to find, appearing on the 2 nd page. Copyright UserWorks, Inc. 2008
- #10: Kristen: [Video Clip: SeniorHealth.gov 1B P1 30:07-32:29] Copyright UserWorks, Inc. 2008
- #11: Kristen: Now, that you had come context your note-taking should improve. Copyright UserWorks, Inc. 2008
- #12: Kristen: Copyright UserWorks, Inc. 2008
- #13: Dana: Now, let’s move on to the third exercise. This time, we’ll be focusing on metrics. It’s important for all members of the research team to come to a consensus and define all metrics prior to data collection. Copyright UserWorks, Inc. 2008
- #14: Dana: Copyright UserWorks, Inc. 2008
- #15: Dana: Again his task is to change the high alter. Copyright UserWorks, Inc. 2008
- #16: Dana: By a show of hands how many people rated it a … We scored this participant task with a 2 (minor difficulty) because the participant tried two different menu items before reaching the correct item, and as defined in the rating scale, that would be a 2. By a show of hands, how many also gave this participant a 2… But, as you can see, that task would have received various ratings had this scale not been established before data collection began. Copyright UserWorks, Inc. 2008
- #17: Dana: Though we didn’t include it in this study’s rating scale, your scale may include assistance or help from the moderator, so you would want to clearly define that as well. Copyright UserWorks, Inc. 2008
- #18: Kristen: Obviously, there are a lot of other factors that go into efficient and effective note-taking, but unfortunately we were unable to cover all of them in detail, but…. Copyright UserWorks, Inc. 2008
- #19: Kristen: These are the overall keys to success [read bullets]…. Thoroughly familiar with the tool: You can either use some type of data collection tool such as Morae or Silverback or you could use some type of Word processor or spreadsheet, or even take notes on paper using a checklist. Add to the last bullet: Participant says one thing and does another Participant thinks they have been successful when they really haven’t Participant doesn’t want to look bad, so bluffs Participant initially struggles, then succeeds, and speaks highly of the product Participant blames self and rates product highly despite disastrous task performance Participant is overly chatty and goes off on tangents Copyright UserWorks, Inc. 2008
- #20: Kristen Copyright UserWorks, Inc. 2008