PLAIN 2013 - Is it really plain? A case (and process) for content testing
2 likes1,690 views
The document discusses the importance of content testing, particularly in creating effective communication materials, emphasizing that clear comprehension and persuasion are essential for reaching target audiences. It outlines a case study involving a brochure about a veteran data program that revealed key facts were not communicated clearly, thereby diminishing its effectiveness. The document advocates for simple and robust testing methods, including comprehension and persuasiveness testing, to ensure content resonates with the intended audience.
PLAIN 2013 - Is it really plain? A case (and process) for content testing
- 1. Is it really plain? A case for content testing Annetta Cheek Center for Plain Language Kath Straub Usability.org | Center for Plain Language PLAIN 2013 | Vancouver CA
- 2. Where we are going …. • A parable • A process PLAIN 2013 | Vancouver CA
- 3. The Parable PLAIN 2013 | Vancouver CA
- 4. PLAIN 2013 | Vancouver CA
- 5. PLAIN 2013 | Vancouver CA
- 6. We identified 10 “must-know” facts. 1. Any veteran can join. 2. When you join, you give access to your VA System electronic health record, including – – – – – – Your vital signs Diagnoses Doctor’s notes Medicines Treatment history Test results 3. VA will use the data starting from the time you entered the system until 10 years after you joined Donate My Data. 4. If you die during the program, the VA will stop using your data. 5. VA will follow the laws about protecting your data. 6. VA will use donated data to test software. 7. VA uses made-up data to test software now. 8. Employees and contractors who build software for the VA will have access to your health data. 9. You can drop out at any time, by sending an email request to a specific address. 10. If you drop out, VA will keep your authorization and withdrawal forms, and stop using your data. PLAIN 2013 | Vancouver CA
- 7. The brochure didn’t test as well as we expected …. Target = 80% recall 1 2 3 4 5 6 7 8 9 10 PLAIN 2013 | Vancouver CA
- 8. What we learned • Some key facts were not clear enough • As a result, the brochure was not as persuasive as desired • If we hadn’t tested, we would never have known. PLAIN 2013 | Vancouver CA
- 9. Why am I telling you this? • We have entered the age of content. PLAIN 2013 | Vancouver CA
- 10. In the Age of Content, there are no guides. PLAIN 2013 | Vancouver CA
- 11. In the Age of Content, there are no guides. PLAIN 2013 | Vancouver CA
- 12. In the age of content, we will have to stop blaming the victim. * Tes?ng shows that the F-‐shaped reading behavior is a symptom of the relevance of content not an determined habit of humans. PLAIN 2013 | Vancouver CA
- 13. Why am I telling you this? • We have entered the age of content. • Comprehension testing sounds harder than it is. (Don’t tell anyone.) PLAIN 2013 | Vancouver CA
- 14. Usability people Content People • Content is not our job, • We are writers, not researchers Usability people its their job PLAIN 2013 | Vancouver CA
- 15. Why am I telling you this? • We have entered the age of content. • Comprehension testing sounds harder than it is. • We were surprised by our own results. Don’t tell anyone this either, ok? PLAIN 2013 | Vancouver CA
- 16. A process (Guerilla content testing) PLAIN 2013 | Vancouver CA
- 17. 3 types of content testing • “Simple” comprehension testing • Confidence testing • Persuasiveness testing • Some logistics PLAIN 2013 | Vancouver CA
- 18. “Simple” comprehension testing 1. 2. 3. 4. PLAIN 2013 | Vancouver CA Agree on the facts Decide which are most important Create a ques?on about each fact Agree on the answers
- 19. The challenge: Good questions are hard to write PLAIN 2013 | Vancouver CA Tips: • Expect to revise your ques?ons several ?mes • People remember m/c strategies from uni (e.g., the longer, specific answer is the right one. • Give par?cipants an alterna?ve to guessing (e.g., “The brochure didn’t say”)
- 20. “Simple” comprehension testing 1. 2. 3. 4. Agree on the facts Decide which are most important Create a ques?on about each fact Agree on the answer 5. Pre-‐test your ques?ons PLAIN 2013 | Vancouver CA
- 21. Comprehension testing: Have a Bake-off Tip: No one version is going to be all beer. By tes?ng mul?ple versions you can compare and contrast which works best (for which audiences.) PLAIN 2013 | Vancouver CA
- 22. “Simple” comprehension testing 1. 2. 3. 4. Agree on the facts Decide which are most important Create a ques?on about each fact Agree on the answer 5. Pre-‐test your ques?ons 6. Use the test to have a bakeoff PLAIN 2013 | Vancouver CA
- 23. Reporting comprehension results PLAIN 2013 | Vancouver CA
- 24. Confidence testing Could you describe the Donate my Data program to a family member or friend? a. I can describe both the idea and the details to someone else, and get them all right. b. I can describe it mostly, but I might get some of the details wrong. c. I can describe the general idea, but not the details. d. I wouldn’t try to describe this to someone else. PLAIN 2013 | Vancouver CA
- 25. Persuasiveness testing Just because I can, doesn’t mean I will. PLAIN 2013 | Vancouver CA
- 26. Persuasiveness testing • Motivators • Blocks/Concerns • Barriers PLAIN 2013 | Vancouver CA
- 27. Mechanical Turk (Mturk). Crowdsourcing science • A parable • A process PLAIN 2013 | Vancouver CA
- 28. PLAIN 2013 | Vancouver CA
- 29. Some MTurk tips • Start with a comprehension test • Embed ‘catch’ questions • Create a ‘panel’ over time • You get what you invest and what you pay for PLAIN 2013 | Vancouver CA
- 30. A case for testing • Testing doesn’t have to be complicated to be robust • Crowd-sourced testing can be quick, cheap and effective. • You don’t really know its plain language to your target audience unless you’ve tested in your target audience PLAIN 2013 | Vancouver CA