Usability Testing Options

Usability Testing: Options to Fit Any Budget Kathy McShea 2008

Usability is: “the effectiveness and efficiency of a system to meet the user’s needs, resulting in user satisfaction and productivity .” Usability is an essential part of all great Web sites Usability Technology Content Usability Engineering is: “ tools , techniques and user centered processes applied to achieve usability.”

Satisfaction = Loyalty Rational Loyalty Are you satisfied? Would you recommend? Would you come back? Emotional Loyalty Confidence This {brand} delivers what they promise This {brand} is a name I can always trust Integrity {brand} always treats me fairly If a problem arises, I can count on {brand} to reach a fair and satisfactory resolution Pride I feel proud to be a {brand} customer {brand} always treats me with respect Passion {brand} is the perfect company for people like me I can’t imagine a world without {brand}

The elephant in the room... VISITORS 85% abandon a site due to poor design 83% leave a site if they have to make too many clicks to find what they are looking for 62% gave up looking for an item while shopping online 40% don’t return due to a negative experience like hard to use content ORGANIZATIONS 50% of sales are lost because visitors can’t find content 51% of sites comply with simple usability principles

Why usability works... Ensures your Web site is: Easy to learn and use Useful to the intended audience Satisfying to use Users can: Easily find information on your site Perform tasks on your site with confidence Understand information (jargon, acronyms, wall of words) Use the information to accomplish their goal Leave satisfied with the experience, recommend and decide to return

Top 3 Benefits Get actionable results to measure progress: are you succeeding or failing? Align with industry best practices – don’t reinvent the wheel Lead with facts not opinions; it is a research-based orientation

Who is your user? User characteristics: Who is the site for? What are the users like? Environmental characteristics: When/where will they access the site Goal and Task Characteristics: Why will they come to the site? What will they do on the site?

What are the right tasks? Assess audience goals using... Surveys Call record data Interviews Search log analysis Field studies Assess business and organization goals... Collect data from leadership Surveys Interviews Review relevant business documents Conduct competitive analysis

What’s the right test? Inspection evaluations Expert Review with scorecard and optional report (1 week) Human performance testing Top Task Tune-up (3 to 4 weeks) Participant recruitment can be done by client Software inspection We can run the LIFT MACHINE test in conjunction with expert review to evaluate accessibility and usability issues Qualitative research Web blueprint package ; facilitating stakeholder and/or audience feedback with competitive benchmarking (4 to 6 weeks)

Expert Review A scorecard maps the user experience to a “value” ladder Prioritize: take it a step at a time , to rise above “discouraged”

Climb the ladder… Awareness : Access to content Tactics : Speed and reliability of the framework must be high Information must be accessible to qualified users Adhere to guidelines for accessibility Marketing your site Satisfaction : Deliver an error-free experience to users – Does it work? Can I recover quickly? Tactics : Contact info for the Web team/Help should be visible in the expected location Broken links are fixed quickly Content is searchable Indexes work as expected

Higher steps… Confidence : Is the site intuitive and learnable? Can I find it? Tactics : Navigation Link behavior Ease of use Readability Packaging Forms Trust: Do you deliver what you promise? Are you transparent? Tactics : Use a privacy policy Populate the “about us” programs on their pages Show content is current by adopting a review schedule Be authoritative, avoid duplication, use related links Graphic display guidelines

We’ve completed over 200 Web Site evaluations using our custom scorecard to measure, usability, 508 compliance, content and policy rules.

Current ES Scorecard Custom Scorecard reviews 36 factors on usability against best practice standards Some items weighted: total score of 100 Providing valuable content is worth 46 points Helping people find stuff is worth 31 points Leveraging feedback is worth 23 points feedback finding content NOTE: The 36 factors could form the basis for future revisions to your editorial guidelines or publishing rules

Publishing Rules We’ve created documentation for standards and publishing rules to govern Web sites with the House Democratic Caucus and HouseNet at the U.S. House of Representatives and the AF Portal Intranet at the USAF.

Expert Review pros and cons Advantages The yardstick you use is best practice standards and evidence-based research Can pave the way for embracing standards across your organization “ Grades” are familiar Actionable report focused on problem spots You can track improvement over time You can compare score with competition Disadvantages Uniform understanding of terms and definitions needed Expert reviewer track record is critical to success You must avoid single point of failure: For two-sets of eyes use software inspection OR second reviewer Binary report means no shades of success Does not involve users Not possible to review features, tasks, functionality

Why More Can be Less.... People judge their own satisfaction based on the peak moments and the end points... SOURCE: Barry Schwartz, author The Paradox of Choice, Why More is Less (2004) Too many choices = paralysis Imagined alternatives = regret Choice = escalation of expectations Today: no excuse for failure

Usability testing process Recruit participants (NLT 8) Create scenarios Conduct the test Bailey’s Usability Testing Environment Remote testing with Go to Meeting Analyze the results Prepare the test report

Usability testing process Create scenarios (user tasks) Usually each person attempts to complete several tasks (up to 7) The scenarios are presented in random order Secure participants Representative of typical users Large enough number to represent typical users

First time At least 95% of typical users will complete a specific task (“purchase a camera”) in less than three minutes At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) within 30 seconds At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) within 45 seconds Objectives

Compared with a Baseline The average time to complete a particular task (“make an airline reservation”) will be 10% faster when using the revised Web site than when using the original Web site At least 95% of typical users will complete a specific task (“purchase a camera”) in 10% less time than the previous test At least 95% of users will be able to find a particular report (“cancer mortality in the U.S.”) in 10% less time than the previous test Objectives

Compared to the Competition The average time to complete a particular task (“make an airline reservation”) will be reliably faster when using our Web site than when using a competitors Web site At least 95% of typical users of our website will complete a specific task (“purchase a camera”) reliably faster than a competitors Web site At least 90% of users will find information on a specific topic (“risks related to taking aspirin”) in reliably faster time than a competitors Web site Objectives

Usability Test Participants Participants 40:05 10 years or more Advanced degree 60-75 Female P16 37:33 5 to 9 years High school 18-39 Male P15 21:25 10 years or more Advanced degree 40-59 Female P14 37:43 10 years or more Advanced degree 60-75 Male P13 54:29 10 years or more Some college 18-39 Male P12 18:55 10 years or more Bachelors degree 40-59 Male P11 45:27 10 years or more Some college 40-59 Female P10 40:26 5 to 9 years Bachelors degree 40-59 Female P9 36:28 10 years or more Bachelors degree 18-39 Male P8 49:51 10 years or more Bachelors degree 40-59 Female P7 38:18 5 to 9 years Bachelors degree 18-39 Male P6 36:47 5 to 9 years High school 40-59 Female P5 29:43 5 to 9 years Some college 40-59 Female P4 36:29 1 to 4 years Bachelors degree 60-75 Male P3 40:16 5 to 9 years Some college 40-59 Female P2 53:50 10 years or more Advanced degree 40-59 Male P1 Test Time Web Experience Education Age Gender Participant

Age and Education Gender Female: 50% Male: 50% Age 18 - 39: 25% 40 - 59: 56% 60 - 74: 19% Education High school: 13% Some college: 25% Bachelor’s degree: 37% Advanced degree: 25% Participants KEY QUESTION: Does the participant profile match your target audience?

We help you target common tasks users may find difficult and use task analysis studies to recommend improvements in usability.

Performance Results Between the old HHS site and the Redesign What Indicated Success 92% 41% Average success rate 88% 8% Your cousin is considering a career in medical research and asked you if HHS offers financial aid to undergraduate students. 94% 71% You want to know what the Fiscal Year 2001 budget for HHS was. 94% 13% You want to know what housing organizations are available to help assist the homeless in your area. 94% 73% You want to know what diabetes is and how you can prevent it. 88% 38% You want to find a nursing home for a relative. Final Prototype Baseline Test Success Rate Scenario Text

Sample Reporting Format Scenario: You want to know what diabetes is and how you can prevent it? Where would you get this information? ____Successful ____Unsuccessful ____Time -Diseases and Conditions -CDC Health Topics Notes/observations Task Completion and Time Pathways

Task Analysis pros and cons Advantages Software automates measures of time, clicks and success Think-aloud method reveals cues on user preferences Good results possible with as few as 8 testers Results helps you focus on what matters most Can be repeated over time Disadvantages Must set up software test correctly to collect data Remote testing format hides visual cues Selecting no less than 8 testers critical Selection bias possible without good screening Deeper review of content and navigation missing One shot test limits impact

Draw the blueprint Endorsement Get it in writing from the organization's top brass: the visionary executive Set goals Baseline current activities and define the "problem" your Web site wants to tackle. Analyze audiences Talk to internal and external audiences and define your market. Benchmark Examine the competition and surface best practices, new ideas. Document requirements Develop list of high-level needs; validate with stakeholder team. Select navigation Lead stakeholder team to decide on primary and secondary navigation with meaningful labels. Brand it Pick a tag line and theme to match your value proposition and fit your market and objectives.

Web blueprint package When? If you are at the start of a site redesign project If you want to avoid an expensive redesign and simply improve navigation If you need a competitive analysis What? Lead a card-sorting exercise to validate navigation Facilitate a session for stakeholder feedback Benchmark capabilities against top competitors

Card-sorting definitions ...a user-centered design method for increasing a system’s findability. The process involves sorting a series of cards, each labeled with a piece of content or functionality into groups that make sense to users or participants Source: Card-sorting: a Definitive Guide by Donna Maurer and Todd Warfel “ ...can provide insight into users’ mental models, illuminating the way that they often tacitly group sort and label tasks and content within their own heads” Source: Information Architecture for the World Wide Web

Card-sorting What is it? Method that generates an overall structure for your information, as well as suggestions for navigation, menus and possible taxonomies Benefits? Explore how users group items – low cost + low tech Develop structures that are logical to users Maximize probability of users finding information

How do you do it? Item names to be categorized printed on note cards, 1 item/ card Participants are asked to group cards in a way that makes sense to them; Participants may be asked to... assign cards to groups and label groups by giving each category names (open sort) sort cards into pre-defined categories (closed sort) Options: Use larger cards and ask whether they understood the term (yes, no or unsure) or how frequently the user uses info. Ask participants to rename cards or think of synonyms clarify taxonomy unmask jargon

Do users want to see information grouped by subject, process, business group or information type? How many potential main categories are there? (typically relates to navigation) What should these groups be called? Which terms/labels are logical and intuitive to users? How similar (or different) are the needs of user groups? Card-sorting answers questions

Card-sorting pros and cons Advantages Easy and inexpensive Reliable User-centric Helps to develop categories that are logical to users Helps to identify items that need to be renamed Helps with terminology Disadvantages Does not consider users’ tasks Not an evaluation technique (like usability testing) Can be time-consuming Sometimes difficult to analyze – tools have limitations Limited number of participants

Facilitated feedback Organization feedback Where is the organization today? (Customers? Suppliers? Competitors? Investors and shareholders?) What are our distinct competencies? What are our vulnerabilities? Where could we easily be attacked? What could we improve easily? What do we feel proud and sorry (rant and rave) about? What are the issues, problems and complaints that we have? Stakeholder feedback How is our world changing? What are the new market conditions and forces? What are the new ways to win? What are others doing? What is threatening us or what are the most urgent things for us to do? Who do we want to be like? Sample

What are YOU ready for? Expert Review with scorecard with scorecard and report Top Task Tune-up with client recruiting participants with participant recruitment Web Blueprint Package

Contact us Kathy McShea Owner & Founder Emerald Strategies, Inc. 202.543.2112 Web Management & Marketing ~ http:// www.emeraldstrategies.net Media Training ~ http:// www.prclinic.com Join my professional network ~ http:// www.linkedin.com/in/kathymcshea

Usability Testing Options

More Related Content

Similar to Usability Testing Options (20)

More from Kathy McShea (15)

Recently uploaded (20)

Usability Testing Options

Editor's Notes