SlideShare a Scribd company logo
Designing Usability Tests to Solve Common Problems
Designing Usability Tests to
Solve Common Problems
Melanie Read, Head of Digital, University of London
Marie Kitney, Head of Client Services, Numiko
IWMW 2018 11 July 2018
london.ac.uk numiko.com
Outline of the session
In this session we will:
• Share experiences about our redevelopment project
• Share some usability best practice we’ve learnt along the
way
• Work together to think about common usability problems we
all encounter
• Start to develop ideas around usability testing
london.ac.uk numiko.com
Welcomes and world’s
worst…
• Talk through in your
pairs/groups about the worlds
worse website experience
• Identify why it was so bad?
• What they could have done to
make it better?
london.ac.uk numiko.com
The brief
More training/guidance on how to use the search on SharePoint would also be helpful
for users.
• Raise the profile of the University of London both in the UK
and overseas
• Attract students and build a lifelong relationship with the
University
• Promote the research outputs and to attract and retain
excellent talent to the SAS research institutes
• Focus on recruitment to address the ever increasingly
competitive distance learning market
• Reduce the 100+ websites the University currently has, which
are a result of departments having their own sites, starting
the journey of moving to one website.
london.ac.uk numiko.com
“Deliver a world class user
experience to 180 countries”
london.ac.uk numiko.com
Designing Usability Tests to Solve Common Problems
What we did
• Analysis of existing web properties
• Stakeholder engagement up to VC level
• User research
• Digital brand
• IA development for a unified site
• Content migration planning
• Content governance and workflow
• Drupal 8 development
• ATLAS integration
• Modular pattern library
• Faceted Course Search using Solr
london.ac.uk numiko.com
london.ac.uk numiko.com
london.ac.uk numiko.com
london.ac.uk numiko.com
The results
• Courses are the second most visited
page after the homepage
• Up 75% visits compared to the
previous year
• Of the top 50 pages, visited 45
pages are about courses or courses
related material
london.ac.uk numiko.com
The results
• A Drupal 8 based web
publishing platform to bring in
100’s of microsites within
London.ac.uk.
• Federated content ownership
with consistent technical
governance.
london.ac.uk numiko.com
…but it wasn’t all plain sailing!
• The project presented a number of usability challenges
• Combining content from 2 websites
• Combining 3 modes of study into a single course search
• Various edge cases e.g. study with distance learning but with LTI support
• User journey of selecting course modules
• Showcasing global reach with local support
london.ac.uk numiko.com
Example persona(s)
• User 1: I live in the UK and want to study at a University in London
• User 2: I live in Singapore and I want to study with some local support
• User 3: I live in the UK and I want to study independently
• User 4: I live in China and I want to study independently
• User 5: I live in Egypt and I want to study at a University in London
• User 6: I don't yet know how or where I want to study
• User 7: I live in China and want to move to Paris to study on campus
• …and many more!
london.ac.uk numiko.com
Task
london.ac.uk numiko.com
Discuss usability problems that your
institutions face
• Share with group – are there commonalities?
• Are there outliers?
• Does anyone have any insights to share in how they’ve overcome
similar?
london.ac.uk numiko.com
Principles of good usability
london.ac.uk numiko.com
What is usability?
“Usability is the extent to which an interactive system can be used by
specified users to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use.”
• Effectiveness = goal completion
• Efficiency = effort/ cognitive load/ time
• Satisfaction = “free from discomfort
london.ac.uk numiko.com
Example
https://www.yorkshirepost.co.uk/news/the-full-list-of-yorkshire-road-
closures-during-the-tour-de-yorkshire-1-9143881
Difference between usability and user experience
• Before use
• During use (usability)
• After use
london.ac.uk numiko.com
Prototypes & Usability Tests
Prototypes serve as basis for:
• Internal design reviews
• Primitive usability testing
Usability tests can be:
• Face-to-face: ideal for touch devices. Recruit and incentivise through social
media
• Remote: moderated sessions via Hotjar or good old phone + conferencing
software. Context of use (natural habitat) is best
• Unattended: avoid those who work in marketing as well as your peers/
colleagues
london.ac.uk numiko.com
Some Useful Tools
Prototypes
Paper, PowerPoint, Axure, InDesign
Usability Testing
Treejack
Hotjar
Silverback
Google Optimize
Visual Web Optimizer (VWO) for A/B testing
Conferencing/ screen-sharing tools such as Skype and Join.me
www.whatusersdo.com
Things to consider
• The master-apprentice model
• Open vs closed questions
• Avoid leading questions, aim to be neutral
• Don’t be too rigid – follow the user
• Make sure user requirements are well articulated first, as
these are the yardstick by which to measure usability later
london.ac.uk numiko.com
Jakob’s Law
“Users spend most of their time on other sites. This
means that users prefer your site to work the same
way as all the other sites they already know.”
- Jakob Nielson.
london.ac.uk numiko.com
Design Patterns
• We used these to solve common recurring problems and to
reduce cognitive load
london.ac.uk numiko.com
Task
london.ac.uk numiko.com
Design a usability test
• Select a few of the more common problems we discussed
earlier
• Draft some tests to assess usability
• Assume it’s a face-to-face usability interview
• Remember, usability = effectiveness, efficiency, satisfaction
• Open/ closed questions, leading/ neutral questions
london.ac.uk numiko.com
Further reading
london.ac.uk numiko.com
Heuristics
Heuristics are generally recognised rules of thumb that help
achieve usability. Some examples:
• Visibility of system status – keep users informed through
feedback
• Match between system and the real world – speak users’
language
• User control and freedom – e.g. undo/ redo
• Consistency and standards – follow platform conventions
• Error prevention – e.g. passwords must contain at least 8
characters
Conduct an heuristic evaluation yourself if you can’t get in
front of users. https://www.nngroup.com/articles/ten-
usability-heuristics/
london.ac.uk numiko.com
Seven dialogue principles
These are highly generalised goals for good usability:
• Suitability for the task
• Self-descriptiveness
• Conformity with user expectations
• Suitability for learning
• Controllability
• Error tolerance
• Suitability for individualisation
london.ac.uk numiko.com
System Usability Scale (to measure
satisfaction)
• I think that I would like to use this system frequently
• I found the system unnecessarily complex
• I thought the system was easy to use
• I think that I would need the support of a technical person to be able to use this
system
• I found the various functions in this system were well integrated
• I thought there was too much inconsistency in this system
• I would imagine that most people would learn to use this system very quickly
• I found the system very cumbersome to use
• I felt very confident using the system
• I needed to learn a lot of things before I could get going with this system
https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
Strongly agree – Strongly disagree.
london.ac.uk numiko.com

More Related Content

PPTX
TLC2016 - Power Using' Blackboard for purely online learners
PDF
Introduction to the Corporate MOOC Conference in HK
PPTX
Open Educational Resources: Benefits and Challenges
PPT
Next steps for excellence in the quality of e-learning (EADTU Paris masterclass)
PPT
Badging Open Content at The Open University
PDF
Creating MOOC materials with learning assessment by screencasting
PPTX
LMS Transitions
PPTX
Intro to MOOCS at Tri-C
TLC2016 - Power Using' Blackboard for purely online learners
Introduction to the Corporate MOOC Conference in HK
Open Educational Resources: Benefits and Challenges
Next steps for excellence in the quality of e-learning (EADTU Paris masterclass)
Badging Open Content at The Open University
Creating MOOC materials with learning assessment by screencasting
LMS Transitions
Intro to MOOCS at Tri-C

What's hot (20)

PPTX
TLC2016 - The use of Collaborate in schools in Northern Ireland – Needs Must!
PPT
Distance Learning Amatyc Access 2008
PPTX
Innovation – a litany of failures!
PPTX
Strategic Education Initiatives
PDF
openSE – open educational framework for computer science Software Engineering
PPTX
Innovative Educational Technology and Educational Infrastructure at MIT
PPT
Digital Innovation Oer Theses Pawlowski20091111
PPTX
MOOCs Back to Reality Oops There Goes Gravity
PDF
Tri-C LMS workshop 20140318 handout
PDF
NetGeners.Net: The ne(x)t generation learner - Skills you need in lifelong le...
PDF
Using a mobile app as a agent for online learning in higher education David S...
PPTX
Team 7 (narrated ppt)
PPTX
Sharon McMeekin: Are we Making Progress in Digital Preservation Training?
PPTX
Mobile Learning - Alastair Clark
PPTX
Adoption of Blackboard in the ELICOS system: Innovating and Augmenting Qualit...
PPTX
Making Adoption Eesy | Tammy Robinson (University of Newcastle) & Mieke Post ...
PPTX
JCCS Online Survey Results
PPTX
Staying relevant in a technology-driven world
PPT
Continuous Improvement in Teaching and Learning – The Community College Open ...
PPT
E learning for education
TLC2016 - The use of Collaborate in schools in Northern Ireland – Needs Must!
Distance Learning Amatyc Access 2008
Innovation – a litany of failures!
Strategic Education Initiatives
openSE – open educational framework for computer science Software Engineering
Innovative Educational Technology and Educational Infrastructure at MIT
Digital Innovation Oer Theses Pawlowski20091111
MOOCs Back to Reality Oops There Goes Gravity
Tri-C LMS workshop 20140318 handout
NetGeners.Net: The ne(x)t generation learner - Skills you need in lifelong le...
Using a mobile app as a agent for online learning in higher education David S...
Team 7 (narrated ppt)
Sharon McMeekin: Are we Making Progress in Digital Preservation Training?
Mobile Learning - Alastair Clark
Adoption of Blackboard in the ELICOS system: Innovating and Augmenting Qualit...
Making Adoption Eesy | Tammy Robinson (University of Newcastle) & Mieke Post ...
JCCS Online Survey Results
Staying relevant in a technology-driven world
Continuous Improvement in Teaching and Learning – The Community College Open ...
E learning for education
Ad

Similar to Designing Usability Tests to Solve Common Problems (20)

PPTX
Usability fail - PSMG March 2012
PPT
An Introduction to Usability
PDF
Precedent #usabilityfail seminar part one
PDF
Usability for Web Designers
PPTX
Enterprise User Experience in Higher Education
PPT
Don’t make me think!
PPT
IWMW 2007: Usability Testing for the WWW
PPTX
Usability fail-20102011
PPTX
User Experience Service showcase lightning talks - December 2018
PPT
Principles of Usability Testing For Historic Newspapers
ZIP
Usability, User Experience and the Internet in the 21st Century
PDF
Usability Testing 101 - an introduction
PDF
How User Testing Can Inform Content - 03/19/12 Content Strategy - Minneapolis...
PPTX
Library Website Evaluation
PPT
User Research on a Shoestring
PDF
Usability testing – Just Do It. Five methods for improving usability in-house
PDF
Web Site Usability
PDF
Web Usability: Making Your Sites More Awesomer
PPTX
Information Experience Lab, IE Lab at SISLT
Usability fail - PSMG March 2012
An Introduction to Usability
Precedent #usabilityfail seminar part one
Usability for Web Designers
Enterprise User Experience in Higher Education
Don’t make me think!
IWMW 2007: Usability Testing for the WWW
Usability fail-20102011
User Experience Service showcase lightning talks - December 2018
Principles of Usability Testing For Historic Newspapers
Usability, User Experience and the Internet in the 21st Century
Usability Testing 101 - an introduction
How User Testing Can Inform Content - 03/19/12 Content Strategy - Minneapolis...
Library Website Evaluation
User Research on a Shoestring
Usability testing – Just Do It. Five methods for improving usability in-house
Web Site Usability
Web Usability: Making Your Sites More Awesomer
Information Experience Lab, IE Lab at SISLT
Ad

More from IWMW (20)

PPT
Look who's talking now
PPTX
Introduction to IWMW 2000 (Liz Lyon)
PPTX
Web Tools report
PPT
Personal Contingency Plan - Beat The Panic
PPT
Whose site is it anyway?
PPT
Open Source - the case against
PPT
IWMW 2002: Avoiding Portal Wars - an MIS view
PDF
What does open source mean for the institutional web manager?
PDF
Library 2.0
PPT
Social participation in student recruitment
PDF
Supporting Institutions in Changing Times: Manifesto
PPTX
IWMW 2019 photo scavenger hunt highlights
PDF
How to Turn a Web Strategy into Web Services
PPTX
Static Site Generators - Developing Websites in Low-resource Condition
PPTX
Looking to the Future
PPTX
Looking to the Future
PPTX
Developing Communities of Practice
PDF
How to train your content- so it doesn't slow you down...
PPTX
Grassroots & Guerrillas: The Beginnings of a UX Revolution
PPTX
Connecting Your Content: How to Save Time and Improve Content Quality through...
Look who's talking now
Introduction to IWMW 2000 (Liz Lyon)
Web Tools report
Personal Contingency Plan - Beat The Panic
Whose site is it anyway?
Open Source - the case against
IWMW 2002: Avoiding Portal Wars - an MIS view
What does open source mean for the institutional web manager?
Library 2.0
Social participation in student recruitment
Supporting Institutions in Changing Times: Manifesto
IWMW 2019 photo scavenger hunt highlights
How to Turn a Web Strategy into Web Services
Static Site Generators - Developing Websites in Low-resource Condition
Looking to the Future
Looking to the Future
Developing Communities of Practice
How to train your content- so it doesn't slow you down...
Grassroots & Guerrillas: The Beginnings of a UX Revolution
Connecting Your Content: How to Save Time and Improve Content Quality through...

Recently uploaded (20)

PDF
Computing-Curriculum for Schools in Ghana
PDF
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
PPTX
Cell Structure & Organelles in detailed.
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
PDF
01-Introduction-to-Information-Management.pdf
PPTX
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
PPTX
master seminar digital applications in india
PDF
2.FourierTransform-ShortQuestionswithAnswers.pdf
PPTX
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
PDF
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
PDF
102 student loan defaulters named and shamed – Is someone you know on the list?
PPTX
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
PPTX
Final Presentation General Medicine 03-08-2024.pptx
PDF
RMMM.pdf make it easy to upload and study
PDF
FourierSeries-QuestionsWithAnswers(Part-A).pdf
PPTX
Institutional Correction lecture only . . .
PPTX
Final Presentation General Medicine 03-08-2024.pptx
PDF
A systematic review of self-coping strategies used by university students to ...
PPTX
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
PDF
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
Computing-Curriculum for Schools in Ghana
3rd Neelam Sanjeevareddy Memorial Lecture.pdf
Cell Structure & Organelles in detailed.
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
01-Introduction-to-Information-Management.pdf
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
master seminar digital applications in india
2.FourierTransform-ShortQuestionswithAnswers.pdf
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
102 student loan defaulters named and shamed – Is someone you know on the list?
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
Final Presentation General Medicine 03-08-2024.pptx
RMMM.pdf make it easy to upload and study
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Institutional Correction lecture only . . .
Final Presentation General Medicine 03-08-2024.pptx
A systematic review of self-coping strategies used by university students to ...
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3

Designing Usability Tests to Solve Common Problems

  • 2. Designing Usability Tests to Solve Common Problems Melanie Read, Head of Digital, University of London Marie Kitney, Head of Client Services, Numiko IWMW 2018 11 July 2018 london.ac.uk numiko.com
  • 3. Outline of the session In this session we will: • Share experiences about our redevelopment project • Share some usability best practice we’ve learnt along the way • Work together to think about common usability problems we all encounter • Start to develop ideas around usability testing london.ac.uk numiko.com
  • 4. Welcomes and world’s worst… • Talk through in your pairs/groups about the worlds worse website experience • Identify why it was so bad? • What they could have done to make it better? london.ac.uk numiko.com
  • 5. The brief More training/guidance on how to use the search on SharePoint would also be helpful for users. • Raise the profile of the University of London both in the UK and overseas • Attract students and build a lifelong relationship with the University • Promote the research outputs and to attract and retain excellent talent to the SAS research institutes • Focus on recruitment to address the ever increasingly competitive distance learning market • Reduce the 100+ websites the University currently has, which are a result of departments having their own sites, starting the journey of moving to one website. london.ac.uk numiko.com
  • 6. “Deliver a world class user experience to 180 countries” london.ac.uk numiko.com
  • 8. What we did • Analysis of existing web properties • Stakeholder engagement up to VC level • User research • Digital brand • IA development for a unified site • Content migration planning • Content governance and workflow • Drupal 8 development • ATLAS integration • Modular pattern library • Faceted Course Search using Solr london.ac.uk numiko.com
  • 12. The results • Courses are the second most visited page after the homepage • Up 75% visits compared to the previous year • Of the top 50 pages, visited 45 pages are about courses or courses related material london.ac.uk numiko.com
  • 13. The results • A Drupal 8 based web publishing platform to bring in 100’s of microsites within London.ac.uk. • Federated content ownership with consistent technical governance. london.ac.uk numiko.com
  • 14. …but it wasn’t all plain sailing! • The project presented a number of usability challenges • Combining content from 2 websites • Combining 3 modes of study into a single course search • Various edge cases e.g. study with distance learning but with LTI support • User journey of selecting course modules • Showcasing global reach with local support london.ac.uk numiko.com
  • 15. Example persona(s) • User 1: I live in the UK and want to study at a University in London • User 2: I live in Singapore and I want to study with some local support • User 3: I live in the UK and I want to study independently • User 4: I live in China and I want to study independently • User 5: I live in Egypt and I want to study at a University in London • User 6: I don't yet know how or where I want to study • User 7: I live in China and want to move to Paris to study on campus • …and many more! london.ac.uk numiko.com
  • 17. Discuss usability problems that your institutions face • Share with group – are there commonalities? • Are there outliers? • Does anyone have any insights to share in how they’ve overcome similar? london.ac.uk numiko.com
  • 18. Principles of good usability london.ac.uk numiko.com
  • 19. What is usability? “Usability is the extent to which an interactive system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” • Effectiveness = goal completion • Efficiency = effort/ cognitive load/ time • Satisfaction = “free from discomfort london.ac.uk numiko.com
  • 21. Prototypes & Usability Tests Prototypes serve as basis for: • Internal design reviews • Primitive usability testing Usability tests can be: • Face-to-face: ideal for touch devices. Recruit and incentivise through social media • Remote: moderated sessions via Hotjar or good old phone + conferencing software. Context of use (natural habitat) is best • Unattended: avoid those who work in marketing as well as your peers/ colleagues london.ac.uk numiko.com
  • 22. Some Useful Tools Prototypes Paper, PowerPoint, Axure, InDesign Usability Testing Treejack Hotjar Silverback Google Optimize Visual Web Optimizer (VWO) for A/B testing Conferencing/ screen-sharing tools such as Skype and Join.me www.whatusersdo.com
  • 23. Things to consider • The master-apprentice model • Open vs closed questions • Avoid leading questions, aim to be neutral • Don’t be too rigid – follow the user • Make sure user requirements are well articulated first, as these are the yardstick by which to measure usability later london.ac.uk numiko.com
  • 24. Jakob’s Law “Users spend most of their time on other sites. This means that users prefer your site to work the same way as all the other sites they already know.” - Jakob Nielson. london.ac.uk numiko.com
  • 25. Design Patterns • We used these to solve common recurring problems and to reduce cognitive load london.ac.uk numiko.com
  • 27. Design a usability test • Select a few of the more common problems we discussed earlier • Draft some tests to assess usability • Assume it’s a face-to-face usability interview • Remember, usability = effectiveness, efficiency, satisfaction • Open/ closed questions, leading/ neutral questions london.ac.uk numiko.com
  • 29. Heuristics Heuristics are generally recognised rules of thumb that help achieve usability. Some examples: • Visibility of system status – keep users informed through feedback • Match between system and the real world – speak users’ language • User control and freedom – e.g. undo/ redo • Consistency and standards – follow platform conventions • Error prevention – e.g. passwords must contain at least 8 characters Conduct an heuristic evaluation yourself if you can’t get in front of users. https://www.nngroup.com/articles/ten- usability-heuristics/ london.ac.uk numiko.com
  • 30. Seven dialogue principles These are highly generalised goals for good usability: • Suitability for the task • Self-descriptiveness • Conformity with user expectations • Suitability for learning • Controllability • Error tolerance • Suitability for individualisation london.ac.uk numiko.com
  • 31. System Usability Scale (to measure satisfaction) • I think that I would like to use this system frequently • I found the system unnecessarily complex • I thought the system was easy to use • I think that I would need the support of a technical person to be able to use this system • I found the various functions in this system were well integrated • I thought there was too much inconsistency in this system • I would imagine that most people would learn to use this system very quickly • I found the system very cumbersome to use • I felt very confident using the system • I needed to learn a lot of things before I could get going with this system https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html Strongly agree – Strongly disagree. london.ac.uk numiko.com