![Collecting Qualitative data
For your report: You require two things qualitative notes and qualitative data
Data to be collected
• Success measures. Indicate in the report how you will determine "success"
for each scenario.
• Time measures. Total time to complete the scenario? Separate time figures
for navigating and for understanding [time spent on content pages]? Time to
recover from an error?)
• Error measures. Measure errors and what you would count as an error.
• Number Pages. Count the number of clicks or pages visited before finding
information.
• Pathway. Indentify users' paths through the Web site.
• Satisfaction Rating. Administer a satisfaction questionnaire to evaluate
impressions and gage satisfaction levels.](https://image.slidesharecdn.com/usabilitytestinguwistacampus-13505122273478-phpapp01-121017171900-phpapp01/85/UCD-and-Usability-Testing-2007-43-320.jpg)
UCD and Usability Testing (2007)
- 2. Overview User Centered Design practices Usability Test?!!! Planning your Usability Test Final word on UT and Usability.
- 4. Usability in Context If web development isn’t driven by an alignment of sustainable technology, user driven content and business driven goals, the corporate Web presence will either fail to meet the organization’s business goals, be troubled by expensive technology challenges or simply ailienate core users! - G.A Buccholz, Web Strategist Usability must be complimented by strategy
- 5. User centered Design A project based design approach that puts the intended users of a site at the centre of its design and development. Know your audience - Lenny Bruce, Comedian
- 6. Typical UCD workflow 1. Define both the business’ and users’ goals & objectives 2. Research site for development, amass necessary content, plan wireframes and develop sitemap 3. Develop and (perform technical) testing of prototypes 4. Evaluating design alternatives 5. Test the site with users 6. Analyze usability problems 7. Propose and integrate solutions.
- 7. Experience Design A design approach that focuses on the creation of experiences that produce desired perceptions, cognition, and behavior among their clients’ “users,” “customers,” “visitors,” or Web Design Tip “audiences.”
- 9. The Designer’s view Most good (front end) designers are not hostile to idea of usability testing. In fact they like the idea of providing sound design to shoot down the ridiculous ideas from clients (internal and external) - David Jarvis, Sr Front End designer
- 10. the usability “Did you knows?” • 45% of users abandon Websites with poor navigability, slow response times and confusing content • 35% of users who experience on a particular site leave that site for a competitor’s site. • 52% of companies make no attempt to measure whether users are successful in finding what they want. • Conversion rates can be increased by 40% by improving user experience on e-commerce sites • Product development cycles can be reduced by 33-50% by incorporating usability engineering methods - stats courtesy Triangle Tech Journal
- 11. User research Methods • Usability Testing • Contextual Interviews • Online Surveys • Individual Interviews • Focus Groups • Card sorting
- 12. What is a Usability Test? 1. A means to measure how well people can use some human made object (e.g web page, a computer interface, a document or device) for its intended purpose 2. A to determine how well people can use a product.
- 13. What is Usability Testing? • It is an iterative test whereby real users personally interacting with the product • It does not involve focus groups, surveys, market trials or even product (in this case, a site) release followed by feedback forms • It is not based on collecting information on opinions but observing and documenting patterns of behaviour user interaction. • It’s a user centred design (UCD) practice
- 14. How does it work A user is presented with a real scenario and he/she is asked to perform a variety of tasks. Observers (including the project team) either viewing in a secondary location or at some latter point must Record notes on what the user says or does. Collect data on paths users take to complete tasks Identify what problems and deadly errors user encountered When and where, users/testers were confused The information is collected and used during the site redesign/revision processes
- 15. Why conduct A Usability Test? Simply to understand the dynamics of your own website To find out how the user interacts with your website (i.e. usage patterns) To identify problems and difficulties users encounter when using their website To identify what works or what needs to be reconfigured.
- 17. UT Project WorkFlow Plan Develop your UT proposal, the guiding document for your UT project. Identify website(s) and/or back end application which will be the focus of the test. Define short term objectives and long term goals. Analyse Segment and Define your target audience, Develop personas, recruit participants. Design Design site, develop scenarios & tasks and post test questionnaire Test and Refine Test, collect data, prepare report and refine or redesign.
- 19. Required Resources Human Resources Usability Test Lead/Organizer - this person will lead all planning activities. Communicates with the various groups within the project team. Involved in recruiting/screening activities. Works with the project team to plan scenarios. Briefs primary team ands observers on their respective roles. Project Team - Department manager, Marketing Manager, web team, front end designer(s) (UI D or graphics artist), back end developer. All actively supporting organization activities. UT Primary Test Team - UT Facilitator and A/V Tech UT Assistant - will assist UT lead, (in some cases) interviews prospective recruits, confirms participation of recruits and invited observers, puts together packages for observers, preparing UT flash cards and initially greeting and introducing testers to facilitator.
- 20. Required Resources Logistics and Tech Requirements Computer lab/ or open room (eg. an office). The room must have reliable internet access. If the room doesn’t have a computer, source a laptop. The room should be large enough to comfortably accommodate, tester, faciltator and AV Support - Make arrangements to have the session tapped A separate or adjoined conference room for observers. It would be better if the footage was viewed in its entirety at some point time after testing. When all observers can all meet. Other Resources Funds
- 21. Set a Budget Expense Item Cost * Recruiting Costs $0.00 - $250 US Video Taping Cost $500 - $900 US Gift or Monetary incentive (x 8 or x 5) < $100US (x 5/x8) Computer lab/Room Rental $?.00 (x no of days required) Stipend for UT facilitator < $50US Stationery (notepads, flash cards, etc) $25.00 - $50 US
- 22. Define AND SEGMENT Your Website’s Target Audience If we want to test the Arthur Lok Jack Graduate School of Business Website, our Target Audience consists of: Usual suspects Prospective Students Current Students Current ALJ Faculty & Staff Prospective ALJ Faculty & Staff Alumni Visitors The often Overlooked suspects Visitors - Visiting Guest Speakers, Local and foreign Faculty, Professionals/Business persons, HR Managers, Goverment Agencies, foreign Research Students, Media Greater UWI Staff - Faculty & Admin
- 23. Conduct a Task Analysis After segmenting your site’s audience: Assemble your working project team - content developers and site developers to do a task analysis. What is it? A Task analysis allows you to learn about your users' goals—what they want to do at your Web site-and your users' ways of working. Task analysis can also mean figuring out what more specific tasks users must do to meet those goals and what steps they must take to accomplish those tasks.
- 25. Personas and Recruits Based on size of test - recruit 5-8 participants. A several studies on UTesting have shown that 5-8 users typically discover 80% of problems within a given interface* Based on your target audience, Develop a user persona for each of your prospective testers (ut participants): User Persona - A Profile which accurately fits the a type of user who commonly use the website The User persona represents one of the main groups of users you have identified as being apart of your target audience.
- 26. Sample Persona for ALJ Graduate school of Business website User: Ms. Diana Smith Target Group Prospective Post Grad MBA Student Profession & Qualifications Media Buying Assistant, Ogilvy Maher, Trinidad. B.A. UVI “I want to find a Key Attributes Masters (business) task oriented, focused on details, career driven programme either here in the Internet Experience Level Caribbean or Intermediate -IM’s and occasionally updates abroad which her facebook and LinkedIn pages but mostly emails, does work related research. Has DSL bests suits my at work, Dial up at home. Seldom uses the chosen career internet at home. track.” Experience with UWI website Limited exposure - visited UWI St. Augustine post grad website twice
- 27. Recruiting Users Finding users/testers: Recruit by placing notices on your website or ads in the paper. Enlist the assistance of people you know in order to find prospective testers. But don’t recruit close family or friends. Use a Screening Questionnaire: An important tool for screening and recruiting qualified participants who will accurately fit your respective users’ persona. For each recruit, recruit 2 alternatives. In case, your 1st choice can not make the test and/or for the purpose of follow up testing. Note: If you are testing a new site NEVER user the original testers for a follow up UT. Contracts & consent forms Once you have chosen your UT testers - ask them to sign a special UWI approved Usability Test Contract of Services and/or Video Tape Release Form - see folder for samples.
- 28. Recruiting Users Don’t be lazy!!!! Recruit persons who are apart of your target audience and are or will be actual users of the site you want to test. The most common result of not recruiting the right people - false results. The members of your IT team/ or web team are most likely not going be the main users of your website so why would you use them for this type of test. Once more, you are not developing for your needs but your target audiences’.
- 29. Organize Your testing Team Primary Testing Team (team involved in administering test or interacting with tester) UT Facilitator, AV Technician, UT Test Assistant Observation Team Usability Test Observers include Website development team - Web Design & Development Team and Marketing (website and content developers) Local web administrators - persons within department responsible for managing and updating content. Subject matter specialists - persons within department responsible for developing and providing web content Managers - IT department, Marketing Department and the Department’s Management (eg ALJ Director, Marketing Officer)
- 30. orient your testing Team Primary test Team Roles The Facilitator Greets and takes care of test participant. Makes the user feel comfortable. Presents scenarios & tasks and asks post test questions Encourages users to verbalise what they are doing (the process) as well as their frustrations and issues Observes and listens carefully the participant Patiently waits for user to complete task. Moves the tester to the next scenario if they become stuck for an extended period. Set a maximum time limit for each scenario. A facilitator should never prompt a user, drop hints or rush the tester Audio/Video Technician Tape session (monitor recording equipment) Note: Before the test, instruct the technician to keep the camera focused on the screen during the testing session, so you can capture what the user is doing.
- 32. Usability Measurements Time on task How long it takes for the user(s) to complete basic tasks? Accuracy How many mistakes did the user(s) make? Recall How much does a person remember afterwards or after a period of non-use? Emotional Response How does the user feel about the tasks completed? The information collected in these 4 areas (during your first test) should be used as a comparative baseline for further testing.
- 33. Tester User: Mr. Lisle Waldron Target group Staff Member Profession AV Tech, School of Education Internet Experience Level Expert
- 34. Planning Scenarios Scenario: A scenario is a short story about a specific user with a specific goal at your site. Scenario Planning Guidelines: • Understanding (information) Consumer behaviour: Include general questions, in your screen questionnaire, to find out more about the user, their surfing behaviours and overall (consumer) interests. This information will be a boon during scenario planning. • Tailor Scenarios!: When developing a scenario it should be designed to resonate with the user/tester, mirroring their own experiences and needs. • Presenting a user with entirely alien scenarios will affect baseline results particularly recall and accuracy.
- 35. Presenting The Scenario Avoid tester fatigue: During a single user testing session, present the user/tester a maximum of 3 scenarios Or you can break a single scenario into three parts Maximum Time given to complete a single task within a given scenario - 10 minutes. Instruct facilitator, at the start of each testing session, to reset the first screen to a page that is offsite, eg search engine or a portal.(eg. Yahoo, MSN or Google). Most users have the home pages of theses websites set as their browser’s default home page. Never start at the pages/site you want to test!
- 36. Setting the Tasks Your task analysis is a key reference point when you are developing the scenario and related tasks for the different users/testers Use Full-scale, task scenarios The scenario presents a goal and steps are included to accomplish the task. A full-scale scenario can either report all the steps that a specific user takes today to accomplish the task, or it can describe the steps you plan to set up for users in the new site. They lay out the steps from the user's point of view rather than from the Web site's point of view. They explain how the site may support the goal-oriented scenarios that you started with.
- 39. Guidelines for UT Observers Be Unbiased - don’t judge users or their methods of finding information. If biased observers should opt out. Focus on what users are doing, their actions and the route they take to seek out the requested information. Be quiet - if you are talking or making comments during a test you aren’t focusing on the user and what they are doing. Take notes - all observers should take careful notes. Your notes will help the project team when they are compiling & evaluating the results. It may seem pointless but note everything that the user is doing. What others may miss, you may catch.
- 41. Post-Test Interview Questions You want to get testers’ impressions of the site after they have interacted with it. You can use: • open-ended questions • closed questions using radio buttons, check boxes, or a Likert scale (rating something on a numerical scale) • a standardized satisfaction questionnaire If you are going to ask open ended questions, ask questions which are not biased, or attempt to bias the user. Example questions: • What is your overall impression of the site? • How would you describe finding what you were looking for on this site today? • What is your impression of the search capability? • What did you like best about the Web site? • What did you like least about the Web site?
- 43. Collecting Qualitative data For your report: You require two things qualitative notes and qualitative data Data to be collected • Success measures. Indicate in the report how you will determine "success" for each scenario. • Time measures. Total time to complete the scenario? Separate time figures for navigating and for understanding [time spent on content pages]? Time to recover from an error?) • Error measures. Measure errors and what you would count as an error. • Number Pages. Count the number of clicks or pages visited before finding information. • Pathway. Indentify users' paths through the Web site. • Satisfaction Rating. Administer a satisfaction questionnaire to evaluate impressions and gage satisfaction levels.
- 46. TIPS for Testing When collecting results of the test give more weight to the actual testing session than post test question interview session. Conduct focused usability tests - i.e. a test which studies a single site or 1 website and directly related sites. These smaller tests reduce your budget, require maximum of 5 users and yield better results. You will also be able to conduct followup tests after redesign Lather. Rinse. Repeat.: Test. Revise. Test. Gather your baseline results, incorporate design changes based on your observations and once revision or redesign is complete. Then Retest.
- 47. About Usability • Mimicry without understanding is not usability. • Usability is a coordinated effort • Usability relies on a team effort • If the vision and design goals of the front end designer (or f/e team)/ webteam are not shared by back-end applications team, the site will ultimately not be very user friendly. • If the plan for the interface is inhibited by limited by an inflexible back end, usability will not be achieved. • If objectives are not communicated/or input not sought or adopted across the project team, the site will undoubtedly not be very user friendly. • Usability requires real-time maintenance. • Usability is always a work in progress. • Consistent usability is key.
- 48. Usability in Context Creating a positive user experience should not be taken lightly. After all it’s about creating business value through leveraging metrics, heuristics, usability testing, copy-writing, information architecture, interface design, information design, workflow and cross browser compatibility. In layman’s terms Making our sites (or applications) useful, usable and user friendly encourages our users to readily return to it and continue using it. Consequently UWI will further be able to build long standing relationships with our clients (end users) through our websites.