UX Research

Editor's Notes

• #50: Livescribe
• #54: Business question: How are users actually viewing your content (in what order, for how long and in what specific pattern or pathways)?Audience question: Have you wondered if critical links, buttons or content messaging is being viewed on a critical page?Description: this methodology is very useful when trying to determine why certain homepage metrics from analytics programs are of concern (not clicking on value proposition element…etc.)The respondent sits at a specialized computer screen and undergoes a simple calibration sequence. Respondent is given a stimulus question or task (active or passive) (show homepage for set period of time 15 seconds)System tracks eye pathways and fixations and produces a data file from that task.Important things to know about eye-tracking Tobii not designed for web sites or changing visual stimulus)This makes actual testing of web navigation (changing from page to page) very complex to actually analyze and not accurateVery effective for single stimulus presentations of fixed durationsExcellent for detailed analysis of home pages or critical landing pages and forms Very insightful for assessing impact of advertizing on homepage visual scanning.
• #55: Business question: Is anyone working on a major (large-scale) site launch or redesign that your company depends on for survival?
• #57: Audience question: “How many of you have a project at the point where it is ready for major commitment? (round A, new release, major new upgrade) I have a web site, software or product and I am about to commit major funding or resources to next phase of developmentDo I have usability problems with the user experience that are basically show stoppers Users cannot download the application Users cannot log in Users cannot set up a profile pageUsers cannot navigate to critical content 1-3 critical tasks in 60 min.
• #59: Business question: Do any of you have new development team that has minimal UX / Usability experience? Is your team employing best practices and are they aware of the key UX and Usability performance issues that an effective solution must meet.Description: A highly experienced usability / UI design expert conducts a structured audit of your system or product and rates the system on best practices and estimated performanceInterview and select an expert who has direct experience in your product category and sectorExpert gathers information from your development team and conducts structured audit based on predetermined best practices. Expert presents findings to your team (sometimes not a happy experience for UX design teams without knowledge of formal UCD methods.Very effective early in development and can be repeated with updates at less cost.
• #60: Business problem: How do I organize information like content, navigation, overall IA so that users understand it?Description: this is an automated version of the classic card sorting studies where you give users a pile of index cards with your content descriptions on them and ask them to sort the cards into groups according to how they relate to the content. Example: If I have a bunch of content categories how do I determine what the groupings are and the high level navigation labels? Lets say you have a site selling women’s underwear and you what to create a navigation structure that matches the users mental model. So do you organize the site by type of underwear on top level and styles, colors, or do you organize the site navigation by life style like (athletic, everyday, intimate) and they by type of article color, and price. Respondents are invited to an online study via email.When they agree they encounter a screen with a list of labels or terms in one column and are asked to sort the terms into groups they find organizationally relevant. When they are finished you can give them another card sort of just finish the study. When the required number of respondents are finished with the card sort you can view the dataCard sorting data is analyzed through the application of cluster analysis (not that easy to understand but very useful)
• #85: OPEN SORT: good for getting ideas on groups of contentCLOSED: Useful to see where people would put the content.
• #91: Card sorting as a method in HCI largely took off during the internet boom of the late 1990’s with the proliferation of website navigation.
• #93: Today it’s one of the most popular methods UX professionals use. In fact, practitioners report using Card Sorting as frequently as task oriented lab-based usability testing.
• #99: Using the cards post-task or post-test.Participant walks table, chooses. Returns to discuss meaning. Log comments for later analysis.
• #108: Usability, according to web design expert Jakob Nielsen, usually includes five aspects:Learnability. In other words, when the user is confronted by a new site, how easily can they figure out what the purpose of the site is and how to use on their own.Efficiency. How quickly can the user perform tasks and navigate on the site.Memorability. If the user returns, what parts and functions of the site are recognizable and don’t need to be relearned. Obviously, you don’t want your site to be hard even for your repeat visitors.Errors. To what extent do features and functions not work as predicted by the user or as designed by the developer.Satisfaction. How happy is the user when they are using your site.
• #109: I should note there that one of my greatest sources of inspiration about what usability is all about comes from this book by Steve Krug. Although it’s aimed at people doing web development in a corporate setting, the techniques are almost all spot on for the library world as well.
• #110: Usability tests are really not such a big deal. Here’s a quick overview of the steps:Come up with a set of 3-5 different tasks that you’ll ask users to perform.Round up some 5-10 volunteers who will act as test participants and then bring them one at a time into a testing area where you’ll observe them as they perform the predetermined tasks.After you’ve observed all the test partipants, you’ll have a pretty good idea of some things that need to be fixed and what things seems to be working OK. After you make the easiest 2-3 fixes, go back and do another round of testing and tweaking, etc.
• #111: But why go through all this trouble? Can’t I just see what’s wrong? Can’t I just ask the staff at my library to identify what needs to be fixed and take care of the problems that way?Well, no, not if you really want to make your user the center of the website’s design. Lots of librarians will want to tell you that the library website is for them, too, and that they are experts on what users want and need. That is true up to a point, but as anyone who has sat on a library redesign team knows, everyone has lots of opinions and insights, many of which are completely at odds with each other.The real user of your library’s website is your patron, your customers, your students, your members, whatever you want to call those folks who don’t work in the library. You want to please those folks more than anyone else.By making an honest effort to let your users determine the look and feel of your website—within reason—you’re likely to have a design that will actually “work” for them. By going directly to your users and recording how they actually use the site, you’ll get information that will ground the endless debates over design matters.Usability testing offers a method based on social science research methods.And, as I see again and again when I run usability tests, the results are always surprising. Your test participants are always going to find things that never occurred to your designers. Most usability experts will echo this experience, I believe.
• #112: OK, so you now have an idea about what service or resource you’re going to test, next you’ll want to think about what actual tasks you want your test participants to do. You’lll want to pick pick tasks that are going to reveal some useful information to you.One obvious place to go looking for tasks are those pages or services that you and your colleagues already know need work, such as your interlibrary loan form or the way that library hours are displayed.Another strategy is to think about what are the most common activities among patrons in your library. Take a look at your site statistics to see what are the most popular pages. Maybe that’s where you want to do your testing.Or maybe you’re about to launch a new page or service. Those are great opportunities for testing.
• #113: OK. So the gear you need is not too complicated. You’ll need a computer….a desktop or a laptop will do. Last year, I had test participants use my smarthphone wen I was testing a mobile web site. If you really want to get serious about user-centered design, you may want to do usability testing on paper sketches that precede any actual website coding. This is perfect acceptable and commonly done. It’s a great way to run tests that will help you get a basic page layout and site architecture problems.You’ll also want to install some screen recording software on the computer that your test participants use. That way, you can capture as a movie all the mouse movements, page clicks, and characters typed; this is really rich data to return to when the tests are done and you are trying to write up your report. I’ll talk in a minute about software options.Another option that has worked for me is to simply have a second person on hand helping you with the test. That person’s sole responsibility is to closely observe the test participant and take detailed notes.Finally, if you have screen recording software, you might as well get a USB microphone that can capture the conversation between the test participant and the test facilitator. You’ll want to encourage the participant to think aloud as much as possible as they perform tasks.
• #114: Here are some sample tasks that I’ve used for various reasons. You can see they are not huge, multistep projects but somewhat straightforward tasks.
• #115: Here are five options for screen recording software. I’ve used CamStudio a lot mostly because it is free and can be installed on any machine. With the others, you’ll get a much richer feature set but will limited by the number of machines you can install it on.
• #116: OK, so if you are doing the tests all by your lonesome (not the best situation but certainly still doable), you’ll be in charge of recruiting test participants, running the test, recording the test (you’ll definitely need screen recording software and a mic), and for prepping the test environment.
• #117: I am so thrilled that just hours ago a librarian in Canada, Amanda Etches, who is known for her work in usability, posted this wonderful picture today captioned, “Guerrilla testing.” This is what appears to be a staff of one person running usability tests from a library cart in a public area in the library.
• #118: If you can get another person to help you out with the testing, you can break up the tasks in rational ways.
• #119: Before you run off and do the tests, I recommend drafting a one-page document that details the protocol for your test. The most important part of the protocol is a script that you’ll type out and read from during the tests. It is essential that when you are delivering the task details to your test participants that you say it in the exact same way to each person and that you never give away details about how to do the task. The whole point of the test is to see how much the participant can do on their own without any expert help; this mimics the real world use of your library site. There’s rarely a librarian looking over your patron’s shoulder as he or she navigates your site.
• #120: Getting properly set up is key. Make sure your mic works, the screen recording software works, and that you’ve cleared the cache in your browser so that no words entered previously in search boxes will appear as your test participants use those search boxes.
• #121: Your script should include an explanation of why you are reading from a script; you don’t want to freak out the participant. You can break the ice before the task performance portion by asking them to talk about websites that they like to use.Make sure you don’t give them a chance to explore your site before doing the tasks. You want to drop them into as much as possible real world scenarios where they have come to the site to do a specific task they had in mind.
• #122: It’s essential that you ask the participant to speak aloud so you can hear them express any frustrations or surprises they’ve had.
• #123: If you can engineer a reward or payment to your testers, that’s great but not essential.
• #124: When the test is done, quickly get yourself to a computer so you can get down any insights you gained. After 2-3 participants, it’s easy to lose track of who did what and what you learned.
• #131: Thousands of users can be tested in multiple languages, locations and time zones in a short period of time. Two approaches:Task – based with a plug in“True Intent” based Quantitative and qualitative data can be collectedCan be extended easily to competitive or longitudinal testingExamples: LEOtrace, Vividence, User ZoomAugments traditional lab testing
• #137: Where it fits in Highly adaptive based on the needs This gives it POWER – it creates the missing link
• #138: Saves time – Very fast, thousands on panels, Money – essence of quick and dirty. Techniques for dealing with noise, unrealistic to be in the lab that long. Combines both qual/quant and attitudes and behavior.
• #145: All the flexibility you need to set up a study and analyzing the dataSignificant support in designing study and analysis. Pricing is all project based – typically very expensive – good choice for a large benchmark study
• #146: - These tools are self-service, with some amount of supportCan do largely the same as RV and Keynote, and becoming more powerful by the day Nice visualizations now Loop 11 is very basic, and UZ is much more flexible fraction of the cost = several hundred to a few thousand.
• #147: - Sort into groups
• #225: Using a monitor like that, the participant can move her/his head freely in a large area, the data accuracy is not lowered by natural head movement.
• #245: This effect can be used to direct attention, for example on an ad. Here two different versions of an ad were eye-tracked. In this case, the model is looking directly at the viewer.
• #246: This effect can be used to direct attention, for example on an ad. Here two different versions of an ad were eye-tracked. In this case, the model is looking directly at the viewer.
• #247: And in this version, the model looks at the product, forming a straight line between her eye and the product name on the package.
• #264: Note: give an example here
• #269: FIX THE PHOTO
• #272: Weird things may come out of this.May only make sense in the future.Don’t just dismiss data.Keep them, it may be useful later.
• #273: Iterate also on your research method