10 tips for a better UX survey
Download as PPTX, PDF77 likes162,617 views
The document presents 10 tips for improving survey design based on experiences and research in user experience (UX). Key recommendations include asking about recent experiences, simplifying questions, and prioritizing the clarity of survey items. Emphasis is placed on user-centered design practices to enhance data quality and the overall effectiveness of surveys.
10 tips for a better UX survey
- 1. 10 tips for a better survey Caroline Jarrett @cjforms UPA2012, Las Vegas
- 2. Story 1: My experience of surveys in UX (up to about 2010) We need lots of data. Your samples are too small. We’re going to do a survey. But, but… er, er, …
- 3. Story 2: Interviews with UX experts, mostly in Australia, changed my views Do you use surveys? Sure. Alongside other user research, all the time.
- 4. Where do we stand on surveys? • Do you do surveys? • Who decides whether to do the survey? 4
- 5. Agenda Survey = Questionnaire + Process Tips about questions 5
- 6. Any ideas? 6
- 7. Any ideas? This is a badly-written question, and most of us could only guess wildly at an answer 7
- 8. The approximate curve of forgetting Long ago High Recent Low Time since event Quality of data Major life event Occasional, salient Unremarkable, repetitive 8
- 9. Tip 1 Ask about recent vivid experience Image credit: Fraser Smith 9
- 10. Looks like bands … 10
- 11. Looks like bands … works like rating scale Howard Schuman and Stanley Presser (1996, reprinted in 1981) Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context Summary at: http://www.rosenfeldmedia.com/books/survey-design/blog/which_is_better_an_open/ 11
- 12. Tip 2 Be careful when offering banded answers Image credit: Caroline Jarrett 12
- 13. How many questions can you find here? And again, how likely would you be to purchase rail tickets through the National Rail Enquiries website or a mobile app from National Rail Enquiries, using a smartphone or tablet device such as the Apple iPad or Samsung Galaxy Tab? 13
- 14. Tip 3 Ask one question at a time Image credit: Shutterstock 14
- 15. What about 1 to 10? Most surveys I see have ratings from 1 to 5 Wouldn’t it be easier to have just two ? 7 points are more accurate! 15
- 16. Likert had several different types of question in his scales Likert, Rensis. (1932). A Technique for the Measurement of Attitudes. Archives of Psychology, 140, 1–55. 16
- 17. You can find an academic paper to support almost any number of points in a range • Krosnick and Presser refer to ~87 papers on ranges • Krosnick, J. A. and S. Presser (2009). Question and Questionnaire Design. Handbook of Survey Research (2nd Edition) J. D. Wright and P. V. Marsden, Elsevier • If you’d like to track down the research yourself: http://comm.stanford.edu/faculty/krosnick/docs/ 2010/2010%20Handbook%20of%20Survey%20Research.pdf or http://bit.ly/KNWlio 17
- 18. Users don’t care much about the number of points. They care more about the questions. What’s the difference between those questions? This survey is too repetitive. 18
- 19. Tip 4 Don’t stress too much about the number of points in your rating scale Picture credit: Flickr - Bill Soderman (BillsoPHOTO) 19
- 20. Well, OK, stress a bit. This scale is downright peculiar. Avoid. 20
- 21. How would you improve this questionnaire? 21
- 22. How would you improve this questionnaire? Lots of ways, e.g. the instructions are too long. But the biggest problem is: many people will want to choose ‘other’ but there isn’t an ‘other’ option 22
- 23. Tip 5 Stress a lot about ‘other’ Design by @RickyBuchanan; t-shirt from nopitycity.com or zazzle.co.uk 23
- 24. Agenda Survey = Questionnaire + Process Tips about process 24
- 25. “Shared reference”: both sides interpret in the same way Service area? Photo credit: Flickr - chaptstickaddict 25
- 26. Tip 6 Interview users about the topics in your survey 26
- 27. Collecting data used to be expensive. A survey was a rare event. http://www.census.gov/history/www/genealogy/decennial_census_records/ 27
- 28. Today, we get surveyed constantly 28
- 29. I respond to a lot of surveys 89 invitations 67 completed surveys 1387questions 29
- 30. They wanted my opinions but it took 6 screens to get to the first opinion question 30
- 31. Could the BBC trim some of these? • Gender • Region you live in • Ethnic origin • Type of Internet connection • Rate how it looks • What do you like about it • What do you dislike about it • Your main reason for visiting the site today 31
- 32. Why not ask one really simple question? 32
- 33. Tip 7 Remove the questions that you don’t need Image credit: Flickr - Enokson 33
- 34. If you ask for answers, you have to read and think about them 34
- 35. Tip 8 Decide how many responses you really need http://www.useit.com/alertbox/20000319.html
- 36. We got so much great data that I put every detail in this report !!!! 36
- 37. Tip 9 Create a draft presentation, early. Really early. 37
- 38. Presser et al 2004: pretesting focuses on a “broader concern for improving data quality so that measurements meet a survey’s objective” Field testing focuses on the mechanics and procedures Cognitive interviewing focuses on the questions Usability testing focuses on interaction http://www.slideshare.net/cjforms/introduction-to-usability-testing-for-survey-research
- 39. Try some cognitive interviewing • Pair up. One person gets to be the interviewer. • Think aloud as you try to answer this question: 39
- 40. Try some cognitive interviewing • Pair up. One person gets to be the interviewer. • Think aloud as you try to answer this question: 'how many windows are there in your house’ (Dillman et al, 2009) 40
- 41. OK, now swap and try this question • Please think about a computer system or web site that you used recently. Now think aloud as you answer this question: 41
- 42. OK, now swap and try this question • Please think about a computer system or web site that you used recently. Now think aloud as you answer this question: ‘I thought there was too much inconsistency in this system’ (from the System Usability Scale, Brooke 1986) 42
- 43. Tip 10 Top tip Test, test, test, test Image credit: infodesign.com.au 43
- 44. Caroline Jarrett twitter @cjforms caroline.jarrett@effortmark.co.uk Slides: http://www.slideshare.net/cjforms Blog posts and more resources: http://www.rosenfeldmedia.com/books/survey-design/ 44