Mixed methods-research -design-and-procedures
- 1. by John W. Creswell, Ph.D. Department of Educational Psychology, University of Nebraska-Lincoln, and Founding Co-Editor, Journal of Mixed Methods Research 1 © Please do not duplicate or use these slides without the express permission of the author. University of Pretoria, October 21, 2008
- 2. Topics Introductions Positioning myself A definition of mixed methods research Steps in the process of designing a mixed methods study 2
- 3. Positioning myself View research as set of interactive components; not always linear Focus on rigorous data collection and analysis Work as an applied research methodogist Trained in quantitative, self-trained in qualitative, lst generation mixed methods writer Serve as a consultant on mixed methods on projects Work on projects in an order not in proposal format; then I reassemble into proper format
- 4. A few more thoughts about myself… Published by Sage Publications, Pearson Education (Merrill Educ.)
- 5. A Definition of Mixed Methods Research
- 6. How would you combine two types of data? Qualitative Text Data This is a sample of a text file of words that might be collected on interview transcripts, observation fieldnotes, or optically-scanned documents. Quantitative Numeric Data 2 3 4 2 5 2 3 1 1 2 3 4 2 3 3 2 1 1 1 5 3 4 1 2 3 1 4 4 5 5 4 1 2 1 4 3 3 5 1 4 2 3 1 5 5 2 2 1 5 3 5 1 3 1 5 3 2 2 5 1 3 2 4 4 3 1 2 4 2 2 4 1 5 5 4 2 1 5
- 7. Framework for viewing perspectives on mixed methods 7 Quantitative Data Qualitative Data Methodology Method Paradigm Perspective Use of mixed methods in other designs Mixed Methods
- 8. A mixed methods researcher… Collects both quantitative and qualitative data “Mixes” them “Mixes” them at the same time (concurrently) or one after the other (sequentially) Emphasizes both equally or unequally
- 9. 9 A definition • Mixed methods research is both a method and methodology for conducting research that involves collecting, analyzing, and integrating quantitative and qualitative research in a single study or a longitudinal program of inquiry. • The purpose of this form of research is that both qualitative and quantitative research, in combination, provide a better understanding of a research problem or issue than either research approach alone.
- 10. 10 Collecting both quantitative and qualitative data Quantitative data Instruments Checklists Records Qualitative data Interviews Observations Documents Audio-visual materials
- 11. 11 Quantitative and qualitative data analysis Qualitative analysis Use text and images, For coding For theme development For relating themes Quantitative analysis Use statistical analysis, For description For comparing groups For relating variables
- 12. 12 Mixing or linking the data Results Connect data: Results Converge data: Embed the data: Quan data Qual data Qual Quan Qual Quan
- 13. Typical situations in which mixed methods is used… To compare results from quantitative and qualitative research To use qualitative research to help explain quantitative findings To explore using qualitative research and then to generalize findings to a large population using quantitative research To develop an instrument because none are available or useful To augment an experiment with qualitative data
- 14. What is the reason for using mixed methods? The insufficient argument – either quantitative or qualitative may be insufficient by itself Multiple angles argument – quantitative and qualitative approaches provide different “pictures” The more-evidence-the-better argument – combined quantitative and qualitative provides more evidence Community of practice argument – mixed methods may be the preferred approach within a scholarly community Eager-to-learn argument – it is the latest methodology “Its intuitive” argument – it mirrors “real life”
- 15. Designing a Mixed Methods Study Preliminary considerations Creating a title Posing a general question Listing the types of data collection and analysis Making explicit your worldview Identifying your research design Drawing a figure of your design Writing a purpose statement Writing research questions Completing a research plan
- 16. Preliminary considerations (before you begin to design) 16 • Research problem • Content – any topics • Fit the problem to mixed methods (arguments) •Access to both qualitative and quantitative data •Background and resources •Receptive audience
- 17. Create a working title Writing the title Short Topic Participants Include the words “Mixed methods” Neutral –neither quan or qual 17
- 18. Pose the general question to be answered Write it as a question Look to see how it is phrased Make sure that it is specific enough and focused (an answerable question) Ask yourself, “when I end the study, what question would like to have answered?” 18
- 19. List your types of data collection (a review) Quantitative data (closed-ended) Instruments Behavioral checklists Records Qualitative data (open-ended) Interviews Observations Documents Audio-visual materials 19
- 20. Activity – List your sources of data Quantitative Sources of Data Qualitative Sources of Data 20
- 21. List your approach to data analysis (a review) Qualitative analysis Use text and images, For coding For theme development For relating themes Design-type Quantitative analysis Use statistical analysis, For description For comparing groups For relating variables Design-type 21
- 22. 22 Postpositivism Determination Reductionism Empirical observation and measurement Theory verification Constructivism Understanding Multiple participant meanings Social and historical construction Theory generation Advocacy/Participatory Political Empowerment issue-oriented Collaborative Change-oriented Pragmatism Consequences of actions Problem-centered Pluralistic Real-world practice oriented Determine your worldview…
- 23. Several stances on philosophy in mixed methods… One paradigm (pragmatism, transformative) (Tashakkori & Teddlie, 2003; Mertens, 2003) Multiple paradigms (dialectic perspective) (Greene, 2007) Linking paradigms to design features) (Creswell & Plano Clark, 2007) Epistemological stance (ontology, epistemology, axiology, methodology) (Guba & Lincoln, 2005) Shared beliefs in a research field (Morgan, 2007) What it is How it informs your study 23
- 24. Make explicit your interpretive lens (theory) Advocacy lens (feminist, racial, ethnic, disability, sexual orientation) (Mertens, 2003) Social science lens (social science theory) Components: What is it Who has used it in your field How it will shape your study (rephrase your guiding research question, if a lens applies) 24
- 25. State your procedures (methods) Procedures for handling your qualitative and quantitative data Sequence – concurrent or sequential or both Emphasis – emphasis on qualitative or quantitative Sometimes both concurrent and sequential phases are used Designs may include more than two phases Think about using a simple, elegant design 25
- 26. 26 QUAN Data & Results Interpretation QUAL Data & Results QUAN Pre-test Data & Results QUAN Post-test Data & Results Intervention qual Process Interpretation Triangulation Design Embedded Design Concurrent Mixed Methods Designs Parsimonious designs (Creswell & Plano Clark, 2007)
- 27. 27 Sequential designs QUAN Data & Results Interpretation qual Data & Results Following up QUAL Data & Results quan Data & Results Interpretation Building to Before- intervention qual QUAN Intervention Trial After- intervention qual Interpretation Exploratory Design Explanatory Design Sequential Embedded Design
- 28. 28 Design Name Equal priority QUAN emphasis QUAL emphasis Concurrent, triangulation QUAL+QUAN QUAN+qual QUAL+quan Concurrent, embedded n/a QUAN(qual) QUAL(quan) Explanatory, sequential, quan first QUANQUAL QUANqual quanQUAL Exploratory sequential, qual first QUALQUAN qualQUAN QUALquan Sequential, embedded n/a (qual) QUAN QUAN (qual) (quan) QUAL QUAL (quan) Design options
- 30. Explanatory sequential design Quantitative Data* Number of cigarettes CES-D6 Qualitative Data* Semi -structured interviews, audio recorded and transcribed * Data collected 10 times over the course of a calendar year for 40 participants Quantitative Analysis Graphic plot of CES - D6 scores over time for each participant Graphic plot of cigarettes/day values over time for each participant Case Selection Qualitative Analysis Descri ption of each case Identification of life events occurring during critical months where smoking increased or decreased Thematic analysis of life events for each case Cross-case thematic analysis Interpretation Why did changes in smoking occur? Quantitative Data Collection (quan) Qualitative Data Collection Interpretation – based on quan ad QUAL results Qualitative Data Analysis (QUAL) Case Selection Quantitative Data Analysis Selected 5 cases maximally varying Identified critical months in which smoking varied Source: Creswell, Plano Clark, Shope, McVea. (in progress) +
- 31. Phase I Qualitative Research - Year 1 Qualitative Data Collection Unstructured Interviews - 50 participants 8 observations at the site 16 documents Qualitative Data Analysis Text Analysis: Using QSR N6 Qualitative Findings Development of codes and themes for each site Phase II Quantitative Research - Year 2 Quantitative Instrument Development Quantitative Test of the Instrument Create approximately a 80-item instrument plus demographics Administer survey to 500 individuals Determine factor structure of items and conduct reliability analysis for scales Quantitative Results Determine how groups differ using ANOVA test Exploratory sequential design
- 32. 32 Process – collection and analysis of qualitative data (before, during, after trial) Embedded research design Experiment Intervention QUAN Data collection Pre-test QUAN Data collection Post-test
- 33. Mixing the quan and qual data Type of Mixing Type of Design Why Mixing Occurs Where Mixing Occurs in Research Process Connecting Sequential One phase builds on the other Between data analysis (Phase 1) and data collection (Phase 2) Merging Concurrent Bring results together After analysis of both quan and qual – typically in discussion Embedding Sequential or Concurrent Either building or bringing results together Either between phases or in discussion after analysis 33
- 34. Methodological issues Concurrent Designs Use strategies to explore contradictory findings Use parallel questions Select sub-sample of quantitative for qualitative Be sensitive to bias from one data collection to the other Sequential designs In Explanatory Design, select qual sub-sample from quan sample In Explanatory Design, consider alternatives for followup qual sampling In Exploratory Design, samples can differ In Exploratory Instrument Design, consider qual data analysis approaches for developing instrument 34
- 35. 35 This mixed methods study will address _________________ (overall content-aim of the study). A triangulation mixed methods design will be used, and it is a type of design in which different but complementary data will be collected on the same topic. In this study, _________________ (quantitative instruments) will be used to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). Concurrent with this data collection, qualitative _______________ (type of qualitative data, such as interviews) will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting both quantitative and qualitative data are to bring together the strengths of both forms of research to ___________________ (e.g., compare results, validate results, corroborate results). Write a purpose statement for a triangulation design
- 36. Write a purpose statement for a embedded design This mixed methods study will address _________________ (overall content aim of the study). An embedded mixed method design will be used, and it is a design in which one data set provides a supportive, secondary role in a study based primarily on the other data set. The primary purpose of this study will use ________________ (quantitative instruments) to test the theory of _____________ (the theory) that predicts that __________________ (independent variables) will influence ________________ (positively, negatively) the __________ (dependent variables or outcomes) for ____________ (participants) at __________ (the research site). A secondary purpose will be to gather qualitative data _______________ (type of qualitative data, such as interviews) that will explore ____________ (the central phenomenon) for _________________ (participants) at _____________(site). The reason for collecting the secondary database is ________________ (e.g., to address different question, to provide support for the primary purpose). 36
- 37. 37 Write a purpose statement for an explanatory design This study will address _______________ (content-aim of the study). An explanatory mixed methods design will be used, and it will involve collecting qualitative data after a quantitative phase in order to explain or follow up on the quantitative data in more depth. In the first quantitative phase of the study, ______________ instrument data to be collected from _______________ (participants) at ___________ (research site) to test _______ (the theory) that explains why ______________ independent variables) relate to the ____________ (dependent variables). The second qualitative phase will be conducted because ______________ (intent of the qualitative phase). In this exploratory follow-up, the ______________ (central phenomenon) will be tentatively explored with ___________ (participants) at _____________ (the research site). The reason for the exploratory follow-up is to _______________ (e.g., to help explain or build upon initial quantitative results).
- 38. 38 This study addresses _________________ (content-area of the study). The purpose of this exploratory sequential design will be to __________________ (e.g., develop an test an instrument, generate a taxonomy). The first phase of the study will be a qualitative exploration of ________________ (the central phenomenon) by collecting ___________________ (types of data) from _________________ (participants) at _____________ (the research site). The second quantitative phase will follow up on the qualitative phase for the purpose of __________(intent of this followup). In the quantitative phase, ___________ (instrument data) will be collected from ____________ (participants) at ______________ (research site). Quantitative research questions/hypotheses will be formulated after the completion of the initial qualitative phase. The reason for collecting qualitative data initially is that _____________________ (e.g., instruments are not available, variables are not known, there is little guiding theory). Write a purpose statement for an exploratory design
- 39. Qualitative research questions Qualitative central question Begin with “what” or “how” Focus on single phenomenon Use exploratory verbs (discover, understand, explore) Non-directional language A general question (allowing participants’ perspectives to emerge) 39
- 40. Quantitative research questions Can be hypotheses or questions State variables – independent, dependent, mediating, covariates Develop from theory Use distinct measures for independent and dependent variables Order variables from independent to dependent 40
- 41. Writing research questions/hypotheses in mixed methods research Write qualitative research questions and write quantitative research questions/hypotheses Also write a mixed methods research question Write these questions separately 41
- 42. A new type of research question: A mixed methods question Three ways to write this question: Methodologically-focused: To what extent do the qualitative results confirm the quantitative results? Content-focused: How do the interviews with adolescent boys support the quantitative results that their self-esteem changes during the middle school years? Hybrid of quantitative and qualitative elements: What results emerge from comparing the exploratory qualitative data about boy’s self-esteem with outcome quantitative instrument data measured on a self-esteem instrument?
- 43. Order the topics for your plan Title Introduction (Problem) Worldview/theoretical lens Audience Purpose Statement Research Questions (Literature Review) Methods Type of Mixed Methods Design (also add definition of mixed methods) Types of Data Collection Types of Data Analysis Sequence/Emphasis/ Mixing Procedures Figure of procedures Anticipated methodological issues Ethical issues anticipated Validity issues Researcher resources and skills References, Appendices 43
- 44. Additional resources Books: Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage. Creswell, J. W. (2008). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage Publications. Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco: Jossey-Bass. Mertens, D. M. (2005). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches (2nd ed.). Thousand Oaks, CA: Sage Publications. Plano Clark, V. L., & Creswell, J. W. (2008). The mixed methods reader. Thousand Oaks, CA: Sage. Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications. Tashakkori, A. & Teddlie, C. (Eds.) (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications. 44
- 45. Additional resources Articles and Chapters: Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed- method evaluation designs. Educational Evaluation and Policy Analysis, 15 (2), 195-207. Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In: A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 209- 240). Thousand Oaks, CA: Sage. Creswell, J. W., Plano Clark, V. L., & Garrett, A. L. (2008). Methodological issues in conducting mixed methods research. In M.M. Bergman (Ed.), Advances in mixed methods research. London: Sage. Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11 (3), 255-274. Ivankova, N. V., Creswell, J. W., & Stick, S. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20. Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed Methods Research, 1, 48-76. Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120-123. 45
- 46. by John W. Creswell, Ph.D. Department of Educational Psychology, University of Nebraska-Lincoln, and Founding Co-Editor, Journal of Mixed Methods Research 46 © Please do not duplicate or use these slides without the express permission of the author. University of Pretoria, October 21, 2008