1 R8360 Guidelines for Reading and Evaluating Quali.docx
- 1. 1 R8360 Guidelines for Reading and Evaluating Qualitative Research Articles 1. Find the research question. It’s typically located at the end of the literature review, right before the Methods section. NOTE – it may not be written as a question, but the intended question is often found within the declared purpose or objective, if the author has not explicitly stated it in question form. a. Describe the phenomenon of interest. Evaluate how consistent it is with what is typically explored in a qualitative study. b. Consider the target group(s)/individual(s)/organizations identified
- 2. in the question. How clearly does the author convey the group of interest in a way that is consistent with qualitative research? c. Review how the question is phrased. Is appropriate qualitative terminology used? How well does this question indicate to the reader as to what type of approach is being used? 2. Check the article title. a. How consistent is the terminology and intent of the title with the research question? 3. Identify the research problem that emerges from the literature review/ background. a. How does the author(s) justify a social problem? b. How thorough is the discussion of research that has been done, and note if the phenomenon or choice of group is not clearly
- 3. and sufficiently justified (e.g., just one or two studies; articles from obscure journals, non-academic sources; or literature that is more than five years older than the study’s published date). c. How appropriate is the research problem to a qualitative inquiry? 4. Identify the research purpose. a. To what extent is the purpose aligned with the research problem (terminology, group of interest, phenomenon of interest)? 5. Identify the approach. a. Where in the article is the qualitative approach identified? b. How well is the approach explained and justified? 6. Consider the description of the sample. a. How well was the inclusion/exclusion criteria described?
- 4. How was the number of cases justified (Mason, 2010)? b. Was a particular sampling strategy identified? Was it correctly implemented? If not, how well were the discrepancies described? c. How well does the sampling strategy fit the approach? d. Did the authors include a description of their efforts to achieve data saturation (see Guest, Bunce, & Johnson, 2006; Mason, 2010)? Theoretical saturation, if appropriate? What was their strategy and how well was it achieved? To what extent does their effort threaten or support the credibility of the study? 7. Consider the recruitment, invitation, and informed consent process. a. Were these elements explained well enough that you could judge this a credible and rigorous process (Guest, 2004)?
- 5. b. To what extent was the informed consent process sufficiently detailed? And, was this sufficient to protect participants from harm and insure confidentiality? 8. Review the data collection tools and procedures. a. Are the actual data collection tools included in the article? If so, to what extent are the questions i. Open-ended? ii. Not leading? iii. Using appropriate, non-technical language? 2 iv. Consistent with the purpose and approach?
- 6. v. Insightful or open-ended so that participants might reveal surprising or unexpected experiences? b. To what extent are the data collection tools consistent (content and procedures) with the identified approach? c. To what extent are the data collection procedures consistent with the identified approach? d. How well are the details, consistencies, and inconsistencies of the procedures explained? Was the detail sufficient that you could judge the procedures as dependable and rigorous? e. To what extent did the authors include discussions of reflexivity in the data analysis process (Mauthner & Doucet, 2003)? 9. Consider the data analysis process. a. Was the data analysis process explained in sufficient detail that
- 7. you as the reader could follow? b. To what extent did the authors follow a published or well- source method of analysis? What was it and was their choice consistent with the approach? c. To what extent did the authors include discussions of reflexivity in the data analysis process? 10. Read the details of the analysis. a. How did the authors summarize the participants in the study? Was there sufficient detail provided to verify that the sampling strategy had been successfully implemented? b. How were the themes or key concepts identified? Was a published strategy followed? Was that strategy consistent with the approach
- 8. of the study? c. How were the results presented? How well did the themes represent the underlying categories or concepts? Were the figures or tables (if included) helpful in understanding the results? d. Did the authors note any unexpected findings or discrepant cases? If yes, what was surprising, if no, does this suggest a potential bias? 11. Review the discussion and how results compared with prior research. a. Was a summary of the results clearly presented in the beginning of this section? b. To what extent were each of the key results interpreted and contrasted with prior literature? How did the authors handle results that challenged or diverged from prior studies? c. To what extent did the study results and conclusions answer
- 9. the research question? d. How credible was the discussion of limitations? e. Do the study limitations weaken the transferability of the study? f. To what extent would the suggestions for future studies be helpful for persons who want to do more research in this area? Are they too broad? Unfocused? 12. Evaluate the conclusion. a. Did the authors convey a clear “take-home” message? b. To what extent were the conclusions appropriate given the study approach, scope, purpose, and limitations? 3
- 10. References Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82. Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 11(3). Retrieved from http://nbnresolving. de/urn:nbn:de:0114fqs100387 Mauthner, N. S., & Doucet, A., 2003. Reflexive accounts and accounts of reflexivity in qualitative data analysis. Sociology, 37, 413–431. Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22(2), 63–75. http://nbnresolving/