Ct lecture 2. questionnaire deisgn
0 likes36 views
This document provides an overview of designing questionnaires for clinical studies. It discusses the purposes and types of questionnaires, as well as important design considerations like question wording and response scales. Specific topics covered include open-ended versus closed-ended questions, filter questions, common problems to avoid in wording, and examples of nominal, ordinal, and continuous response scales like Likert scales and visual analogue scales. The goal is to provide guidance on creating valid and reliable questionnaires to collect standardized data from clinical research participants.
Ct lecture 2. questionnaire deisgn
- 1. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Design of Questionnaire With thanks to Dr. Sarah Dennis, UNSW School of Public Health and Community Medicine
- 2. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Consider this story … Two priests, a Dominican and a Jesuit, are discussing whether it is a sin to smoke and pray at the same time. After failing to reach a conclusion, each goes off to consult his respective superior. The next week, they meet again. The Dominican says “Well, what did your superior say?” The Jesuit responds, “He said it was all right.” “That is funny,” the Dominican replies. “My superior said it was a sin.” The Jesuit says, “What did you ask him?” The Dominican replies, “I asked him if it was all right to smoke while praying.” “Oh” say the Jesuit. “I asked my superior if it was all right to pray while smoking”.
- 3. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 What we will learn? • What is a questionnaire and purposes • Types of survey questionnaires • Rules of questionnaire design
- 4. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 What is a questionnaire • An instrument (form) to – Collect answers to questions – Collect factual data • Gather information or measures • A series of written questions/items in a fixed rational order
- 5. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Why use a questionnaire • A well designed questionnaire: – Give accurate and relevant information for research – Minimizes potential sources of bias – Will more likely be completed As simple and focussed as possible
- 6. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Use of questionnaire Ideal use of questionnaire: • There is a large sample • You want fairly straightforward information • You want standardised data from identical questions • You are more interested in what occurs rather than why or how
- 7. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Limitation of questionnaires • Can be superficial - difficult to capture the richness of meaning • Cannot deal with context - information is collected in isolation of environment • Information is not causal - cannot attribute cause-effect relationships • Information is self-report - which does not necessarily reflect actual behaviour
- 8. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Types of surveys • Mail – cheapest, wide coverage, standardised, low response rate (?) • Telephone – medium cost, wide coverage, medium response rate, standardisation depends on interviewer • Face to face interview – most expensive, coverage depends on personal contact, highest response rate
- 9. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Designing a questionnaire • Is a questionnaire appropriate? • Identify the resources that are available • Decide what information you need • Select items for inclusion • Design the individual questions
- 10. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Developing questions • Search the literature – Bibliographic databases (eg: Medline;Cinahl; Psychinfo) – Citation searches of key articles • Preliminary research – Focus groups – Key informants interviews
- 11. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Types of information • Knowledge - what people know • Opinions, attitudes, beliefs, values - what people think about an issue • Behavior - what people do • Attributes – what are people’s characteristics • Remember – based on self-report
- 12. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Knowledge What is the recommended interval between eye checks for patients with uncomplicated diabetes? ☐ 6 months ☐ 1 year ☐ 2 years ☐ Not sure
- 13. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Opinions What do you think are the major issues affecting education standard in Vietnam at the moment? ________________________________ ________________________________
- 14. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Behavior Have you developed a care plan for any of your patients? ☐ Yes ☐ No
- 15. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Attributes When did you graduate from university? _______________________
- 16. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Types of questions Open-ended questions • What? Why? How? • No predetermined responses given • Able to answer in own words • Useful exploratory research and to generate ideas • Flexible • Requires skill in asking questions and interpreting results • Answers can lack uniformity and be difficult to analyse
- 17. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Open-ended questions What do you think about the quality of discharge summaries from the ED at hospital X? ________________________________ ________________________________ ________________________________
- 18. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Close-ended questions • Designed to obtain predetermined responses (Yes/No; True/False; strongly agree-strongly disagree, etc..) • Easy to count and analyse • Easy to interpret • May not have catered for all possible answers • Questions may not be relevant or important
- 19. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Close ended questions The discharge summaries from hospital X allow me to provide adequate care to my patients: Closed Closed- -ended questions ended questions The discharge summaries from hospital X allow me to provide adequate care to my patients: Strongly Agree Neutral Disagree Strongly agree disagree
- 20. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Close ended questions Close Close- -ended questions ended questions Please rate the quality of the discharge summaries from hospital X: Poor Fair Good
- 21. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Filter questions • Filter questions useful to ensure respondents only answer relevant parts of questions
- 22. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Filter questions • Unfiltered – If you use a medical software program, which one do you use? • Filtered – Do you use a medical software program? ☐ No - jump to next question ☐ Yes - which one?
- 23. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Filter questions • Skip in questionnaires more easily managed if these are computer-assisted • Consider including ‘not applicable’ category: – In the past week, how often have you used MEDLINE: ☐ Not at all ☐ At least once ☐ More than once ☐ I do not have access to Medline
- 24. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Getting the question right • Question wording – Questions need to be clear, simple and precise – Poorly written questions lead to ambiguity and misunderstandings and can be wasteful • Response – open, closed, what type of response set will you use?
- 25. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Leading questions: Do you prefer being examined by a doctor of your own sex?
- 26. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Leading questions: Do you prefer being examined by a doctor of your own sex? Would you rather be examined by a: Male doctor Female doctor Either/doesn’t matter
- 27. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Vague questions: Taken altogether, how happy are you with your stay in hospital? Have you seen a doctor during the past year?
- 28. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Vague questions: Taken altogether, how happy are you with your stay in hospital? Overall, how would you describe the care you received in hospital?
- 29. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Have you seen a doctor during the past year? In the last 12 months, have you visited a general practitioner? How long has it been since you last visited a general practitioner? (within the last month, between 1 and 12 months ago, more than 12 months ago)
- 30. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Biased or value-laden questions: Do you think evidence-based medicine is a waste of time? Do you think fizzy drinks are bad for teeth?
- 31. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Biased or value-laden questions: Do you think evidence-based medicine is a waste of time? What do you think of evidence-based medicine?
- 32. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Threatening questions How often do you smack your child? Do you know enough about treating patients at risk of stroke?
- 33. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Threatening questions How often do you smack your child? How often do you use each of the following to discipline your child? Do you know enough about treating patients at risk of stroke? How would you rate your knowledge of X for treating patients at risk of stroke
- 34. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Double-barrelled questions – two concepts in one question Have you had a neck ache or a back ache since your last visit?
- 35. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Double-barrelled questions Have you had a neck ache or a back ache since your last visit? Since your last visit, have you had any of the following symptoms (tick as many that apply): ☐ Neckache ☐ Backache ☐ Headache
- 36. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Negative questions • avoid using negative wording ‘not’, ‘rarely’, ‘never’, or words with negative prefixes ‘in-’, ‘im-’, ‘un-’. Doctors should not be required to see patients outside surgery hours: agree / disagree
- 37. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Negative questions Doctors should not be required to see patients outside surgery hours: agree / disagree Doctors should be required to see patients outside regular hours: Agree/Disagree
- 38. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Complex questions On a scale of 1 to 10, please rate for each of the 12 categories listed below, your level of knowledge, confidence and experience.
- 39. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Common problems with wording Complex questions On a scale of 1 to 10, please rate for each of the 12 categories listed below, your level of knowledge, confidence and experience. Please complete the table below about your level of knowledge, confidence and experience in each of the following areas. Please complete the table below about your level of knowledge, in each of the following areas.
- 40. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Sphere Sphere Sphere
- 41. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Close ended questions are usually followed by a set of responses • Choose type of scale: – nominal – ordinal – continuous (summed items with ordinal response scale)
- 42. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Nominal Are you ☐ Man ☐ Women What is your current marital status ☐ Single ☐ Married ☐ Widowed ☐ Divorced ☐ Separated ☐ Other
- 43. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Limited choices of responses, lack of consistency in what a yes/no, agree/disagree response means Do you have trouble climbing stairs? • Attitudes and behaviours lie on a continuum To what extent do you experience difficulty when climbing stairs in your house? – None – A little – Quite a bit – A lot – I do not have stairs in my house
- 44. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Ordinal Cancer stage: ☐ Localised ☐ Regional ☐ Metastatic
- 45. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Ordinal What is the highest level of education you have reached: ☐ Did not complete primary school ☐ Completed primary school ☐ Up to, but not including year 10 ☐ Completed year 10 or equivalent HSC or equivalent ☐ TAFE education ☐ University
- 46. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Types of ordinal/continuous response scales • Visual analogue scales (VAS) Overall, how much pain have you experienced in the previous hour? Responses Responses Types of ordinal/continuous response scales – Visual analogue scales Overall, how much pain have you experienced in the previous hour? No A lot of pain pain
- 47. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Provide adjectives for points along the line (adjectival scales) Responses Responses Provide adjectives for points along the line (adjectival scales) Painless Some pain Very painful
- 48. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Semantic differential scales (bipolar scales) My illness is: Responses Responses Semantic differential scales (bipolar scales) My illness is: Painful Painless Serious Mild
- 49. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Likert scale: rate agreement with a series of statements. To what extent do you agree or disagree with each of the following statements: ☐ Strongly Agree ☐ Agree ☐ Neither ☐ Disagree ☐ Strongly disagree
- 50. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • How many steps/boxes should there be? – five to seven response categories ideal • People averse to extreme ends of a scale – avoid absolutes eg; almost always vs always, almost never vs never – add throw away categories at either end
- 51. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Responses • Should there be an even or odd number of categories – not an issue if your scale goes from ‘not at all’ to ‘very much’ (unipolar scales) – If your scale is bipolar (eg: strongly agree to strongly disagree), decide whether you want a ‘neutral’ point
- 52. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses • Effort required to answer questions – Eg: During your last consultation with your doctor, did the doctor discuss medications to help lower your blood pressure? – What is meant by discuss?; relies on recall of discussion – Many respondents will tick a response that is ‘satisfactory’; that is, to just ‘tick a box’.
- 53. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses • Fatigue/boredom/disinterest – agree with everything – just say ‘don’t know’ – always choose first response – ‘randomly’ respond without considering the question • Social desirability • Aversion to extreme ends of the scale
- 54. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Minimising fatigue/boredom • Keep task simple – Eg easier to recall more recent events • Keep words short and easy to understand Maintain motivation of participants – ensure task is relevant • Ask people to justify their responses
- 55. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Minimising social desirability • is difficult • instruct that it is ok not to know something
- 56. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Aversion to extreme ends of scale • Avoid absolutes ‘never’, ‘always’ • Expand number of categories by including throw away categories at the end: – never, almost never, infrequently, sometimes, usually, almost always, always
- 57. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Minimizing ceiling effects • Average response doesn’t have to be middle response Problems with responses Problems with responses Minimising ceiling effects ‘Average’ response doesn’t have to be middle response Unsatisfactory Average Above Very Outstanding average much above average
- 58. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Halo effects • often occur when evaluating individuals • judgements made on aspects of a person’s performance influenced by overall impression of the person • a global summary just as informative
- 59. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Framing effects Scenario 1: With treatment, your chance of dying from cancer of the big toe is reduced by about 34%.
- 60. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Framing effects Scenario 2: • Without treatment, your chance of dying from cancer of the big toe is 6 out of 1000. • With treatment, your chance of dying from cancer of the big toe is 4 out of 1000.
- 61. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Framing effects Scenario 3: • With this this treatment, 500 adults would need to be treated to prevent one death from cancer of the big toe.
- 62. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses • Framing outcomes in terms of ‘survival’ or ‘dying’ will also influence responses • Be careful how you ‘frame’ your questions; aim for neutral terms • If unavoidable, present all relevant information
- 63. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses Order effects • May be more likely to endorse first or last response • Preceding questions may influence responses to questions that follow
- 64. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Problems with responses • Randomize order of response sets between individuals • Randomize order of items within questions • May be possible to randomise order of questions • Don’t always present ‘positive’ or ‘negative’ sounding response first • Easier to randomise in computer-assisted interviews than paper & pen questionnaires
- 65. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Ordering questions • Sequence should be logical to the respondents and flow smoothly from one question to the next • Questions tend to flow from: – general to specifi – impersonal to persona – easy to difficult
- 66. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Validity and reliability • Validity: – question measure what you claim it measures – problem with self-report • Reliability: – results are reproducible or consistent with similar groups of respondents, over time and when other people administer the questionnaire
- 67. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 A pointless question … Have you ever been or are you now involved in espionage or sabotage, or in terrorist activities or genocide or between 1933 and 1945 were you involved, in any way, in persecutions associated with Nazi Germany or its allies? ☐ Yes ☐ No From I-94 form US Citizenship and Immigration Services
- 68. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Summary DO • Use simple wording • Be brief • Be specific
- 69. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Summary DO NOT • Be vague • Talk down to respondent • Use abbreviation • Use objectionable questions • Be redundant
- 70. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 Keep your questionnaire SHORT and the questions SIMPLE, FOCUSSED and APPROPRIATE
- 71. Workshop on Analysis of Clinical Studies – Can Tho University of Medicine and Pharmacy – April 2012 References • Foddy W (1993). Constructing questions for interviews and questionnaires: Theory and practice in social research. Cambridge University Press, Melbourne. • Oppenheim AN (1992). Questionnaire design, interviewing and attitude measurement. Pinter Publishers, London • Schuman H, Presser S (1996). Questions and answers in attitude surveys: experiments on question form, wording, and context. Sage Publications, San Diego. • Streiner DL, Norman GF (1995). Health Measurement Scales: a practical guide to their development and use, Oxford University Press, Oxford, 1995.