Need a little usability?
Download as PPTX, PDF8 likes1,546 views
The document discusses usability testing in the context of voting, exploring various factors that affect voter accuracy, comprehension, and preferences regarding ballot design and instructions. It outlines methodologies for conducting usability tests, including project details and findings related to voter interaction with different ballot layouts and instructions. Additionally, it emphasizes the importance of participant respect, avoiding bias in questioning, and the need for comprehensive data collection to enhance voting usability.
Need a little usability?
- 1. Need a little usability? What you can learn from usability testing Whitney Quesenbery Dana Chisnell Usability in Civic Life
- 2. 2 What questions do you have?
- 3. 3 Usability testing answers questions, so we have to understand the question we are asking If we want to know: Which ballot design helps voters be more accurate? We are asking a question that is Comparative, quantitative (errors, time) (and why they made those mistakes) If we want to know: Do voters understand how to mark their ballot? We are asking a question that is abut Mental models, variability, qualitative insights (and how prevalent each model is)
- 4. 4 Helpful models for thinking about usability A’s Ability, aptitude, attitude (what voters bring to the election) E’s Efficient, effective, engaging, error-tolerant, easy to learn (dimensions of usability) Use Useful, usable (operable), desirable (how we define success)
- 5. 5 The recipe for planning a usability test Location and context: formal informal Recruiting: defined opportunistic Activities: instructed tasks free tasks Questions: structured unstructured Data collection: observation task/data only Results quantitative qualitative
- 7. 7 Which instructions work better? Traditional ballot instructions vs. plain language version About this project 45 participants 3 lab setting, 8 days 2 moderators Instructions for voting Careful observation Satisfaction survey We learned about Voter accuracy Order effects Voter preference Results Statistical analysis Participant preference
- 8. 8 Which layout works better? Left aligned vs. right aligned names? About this project 100 participants 2 locations, 1 day 4 moderators + 6 others Instructions for voting Minimal observation We learned about Voter preference Time to vote Number of errors Results Statistical analysis of marked ballots
- 9. 9 What do voters do with unusual options? How do voters react to “double-vote” options? About this project 200 participants 5 locations, 1 day 12 moderators + 5 others Self-defined tasks Minimal observation We learned about Voter awareness of party Types of marks and errors Need for voter ed. Results Inspection of ballots for marking patterns
- 10. 10 What problems might voters have with a new ballot? What problems might voters have with a new ballot design? About this project 1 of 5 similar tests 10 participants 1 locations, 1 day 2 moderators + 1 others Self-defined tasks Observation Post-task questionnaire We learned about Navigation on the ballot Instructions Results Found pattern of errors for one interaction Time on task
- 11. 11 What do voters think about different systems? Which accessible voting system do voters prefer? About this project 496 surveys; multi-disabilit 11 systems Multiday “voting fair” They learned about Voter assessment of usability Voter comments about accessibility Differences by disability Results Statistical analysis survey results Qualitative analysis of comments
- 12. 12 Do voters understand how their votes are counted? Can voters explain and act on that understanding? About this project 20 participants 4 locations, 1 day 8 moderators Self-defined tasks Minimal observation We learned about Navigation on the ballot Instructions Mental models of counting Whether there’s a problem Results Qualitative insights
- 13. 13 Do voters understand a new voting system? What is their reaction to new procedures? About this project Focus group Mock election 1 moderators They learned about Attitudes and reactions Results Qualitative insights
- 14. 14 Things to watch out for when planning a usability test
- 15. 15 Treating participants with respect Language and cultural issues Managing context Participant protections and informed consent
- 16. 16 Avoiding bias Leading questions Closed vs. open questions Sampling bias Moderator influence Observer (or moderator reactions)
- 17. 17 Thoughts?
- 18. 18 Whitney Quesenbery whitney@wqusability.com Dana Chisnell dana@usabilityworks.net Field guides to ensuring voter intent http://tinyurl.com/ensuringvoterintent Usability in Civic Life http://usabilityinciviclife.org Civic Design http://civicdesigning.org LEO Usability Testing Kit http://www.usabilityinciviclife.org/voting/leo-testing-kit/
Editor's Notes
- #12: On April 4th, 2005, citizens with disabilities from all over the state tested voting systems and completed surveys documenting their reactions to each system. A total of 496 surveys were completed.The survey results on the following page will be one of several factors used to help the Oregon Secretary of State decide which voting system or systems to purchase to give citizens with disabilities the opportunity to vote privately and independently. Survey was a modified SUS – 12 questionsI would like to use this voting system in an election. There were too many steps in using this voting system.I thought this voting system was easy to use. The instructions for this voting system were difficult to understand. The buttons or touch screen were easy to use. It was hard to move around the ballot with this system. I think that most people could learn to use this voting system very quickly.I found this voting system awkward or difficult to use. I felt very confident that my vote was cast correctly with this voting system. I would need help each time I used this voting system. The ballot text was easy to read or hear. Casting my vote was easy.