Survey questions and measurement error
Download as PPTX, PDF0 likes1,148 views
The document outlines the importance of standardizing survey questions to ensure consistent and accurate responses from participants. It discusses various components of measurement error and theories on the response process, including a four-stage model. Additionally, it presents different evaluation methods for questionnaires, categorized into quantitative, qualitative, and mixed methods approaches.
Survey questions and measurement error
- 1. Asking survey questions: identifying problems Debbie Collins
- 2. How many hours did you work last week? Consider the following survey question…
- 3. 3 Consider the following survey question… How many hours did you work last week?
- 4. Survey Question Standardisation “The goal of standardization is that each respondent be exposed to the same question experience, and that the recording of answers be the same, too, so that any differences in the answers can be correctly interpreted as reflecting differences between respondents rather than differences in the process that produced the answer.” Fowler F. J. & Mangione T.W. 1990. Standardized Survey Interviewing: Minimizing Interviewer-Related Error. ASRMS Vol: 18. Sage.
- 5. Components of measurement error Questions misunderstood that cannot be answered (accurately) that respondents will not answer Administration question not read as worded record answers inaccurately
- 6. Theories on Response Process • Four stage model (Tourangeau, 1984) – Comprehension of question – Retrieval from memory of relevant information – Judgement and estimation process – Response process; mapping answer to response options
- 7. Questionnaire evaluation methods Quantitative • Behaviour coding • Analytical methods e.g. Item Response Theory, Latent Class Analysis • Split ballot experiments • Field piloting Qualitative • Focus groups • Cognitive Interviewing • Interviewer debriefing • Respondent debriefing • Usability testing Desk based review • Questionnaire Appraisal System • Expert review Mixed methods • Eye tracking with cognitive interviewing • Field piloting with respondent debriefing/web probing • Spilt ballot experiment with cognitive interviewing