M fitt dissertationproposaldefense_redux
- 1. The Doctoral Dissertation Literature Review: An Exploration of the Boote & Beile Literature Review Scoring RubricDept. of Instructional Technology& Learning SciencesUtah State UniversityM. Harrison Fitt, B.S., M.S.
- 2. Welcome
- 3. Why Should We Care? Education research doctoral programs sometimes fall short in preparing future scholars
- 4. To improve education research, future scholars need to be taught the habits of the mind, heart, and hand (Shulman, 2005) unique to their discipline.
- 5. Lit. review skills embody a scholar’s ability to contribute to the field in an interactive, reflective, and generative manner
- 6. Poor lit. reviews frequently lead to poorly constructed research; good reviews reduce methodological errors & duplication of former research
- 7. Initial research points to low overall quality of dissertation literature reviews in education researchSo What’s the Problem? The dissertation provides insight into PhD candidates’ preparation for independent work
- 8. There is a small but growing body of research examining the dissertation literature review
- 9. As part of that body of research, the Literature Review Scoring Rubric (Boote & Beile 2005) is an assessment tool for dissertation literature reviews
- 10. The rubric is being used as a pedagogical tool & has been included in education research textbooks
- 11. However, Boote & Beile’s initial research has not been replicated nor is there adequate reliability and validity evidence to warrant such widespread adaptationWhy use the Boote & Beile (2005) Literature Review Scoring Rubric? What is the rubric?
- 12. Who is using it?
- 13. How is it being used?
- 14. Why is it a useful framework? How Do These Studies Address the Problem in Unique Ways? Article I: Replication & Reliability
- 15. Article II: Validity & Refinement
- 16. Article III: First foray into a training needs analysisAssessing the quality of Doctoral dissertation literature reviews in Instructional TechnologyArticle 1
- 17. Problem StatementThe Boote & Beile (2005) rubric was created to evaluation dissertation lit. reviews in education research
- 18. It is being used widely in a variety of fields for a variety of purposes
- 19. However, in their initial study, the rubric was only applied to the literature review chapter of quantitative dissertations
- 20. We don’t know if the rubric works well for specific fields of study or for other types of dissertations
- 21. Additionally, no reliability data is availablePurpose & Significance of Study Applies the Literature Review Scoring Rubric to a more narrow field of inquiry within education research
- 22. In addition to Chapter 2, examines the dissertation as a whole
- 23. Examines dissertations that incorporate a range of methodologies
- 24. Analyzes the inter-rater reliability of the rubric Findings Mean score for the Instructional Technology (N=27) dissertations was 19.96 (SD =3.16) out of 37
- 25. Little difference exists between the scores for chapter 2 and the overall dissertation when examined as a whole
- 26. Low inter-rater reliability (.344) Article IIThe Boote & Beile Literature Review Scoring Rubric: A content validity study
- 27. Problem Statement The Literature Review Scoring Rubric (Boote & Beile, 2005) is a performance assessment for doctoral dissertation literature reviews
- 28. It is gaining widespread use across disciplines and research methods
- 29. However, only one study reports reliability and no studies examine validity evidence
- 30. More empirical research about the rubric & the constructs it measures is needed to justify its broadening usePurpose & Significance of Study Will contribute to the validity evidence of the Literature Review Scoring Rubric
- 31. Insights and recommendations will be applied to a revised version of the Literature Review Scoring Rubric
- 32. May provide valuable information to aid others in the decision to use the rubric
- 33. May contribute to the understanding of the performance expectations of the dissertation literature reviewExpected FindingsWill have a high content validity index for the 12 criteria
- 34. Content validity index for performance levels will be lower
- 35. These expectations are reasonable as faculty seem to disagree on the important of a dissertation literature review (Lovitts, 2006; Zaporozhetz, 1987)
- 36. Findings will be applied to a revised form of the rubric
- 37. Results will be submitted to Research in Higher EducationTraining Future Education Scholars: Literature Review Skills, Textbooks, and the Doctoral StudentArticle III
- 38. Problem StatementStudies using the Literature Review Scoring Rubric reveal problems with quality of lit. reviews
- 39. Understanding how those skills are being currently taught is an important part of improving quality
- 40. Nearly all doctoral students in the US have to take a research methods course in which they are exposed to literature review skills in their textbooks
- 41. Examining these textbooks provides an entrance point into understanding what students are being taught about dissertation literature reviewsPurpose & Significance of StudyA direct answer to Boote & Beile’s (2005) call to extend research on how doctoral students are taught literature review skills
- 42. To provide an understanding of what literature review skills students are being exposed to in their education research textbooks
- 43. Will be the first time textbooks are empirically investigated using specific performance criteria of dissertation literature reviews
- 44. May yield useful information for instructors who are selecting textbooksExpected Findings Based on the pilot study and anecdotal reports, low overall scores for textbooks are expected
- 45. Search techniques will be covered
- 46. Less attention will be paid to the rhetorical devices of literature reviewsQuestions