Managing Quality In Qualitative Research
7 likes1,744 views
The document discusses the management of quality in qualitative research, emphasizing the need for standards, criteria, and guidelines to address internal and external quality challenges. It highlights various methods of triangulation to improve research quality, including data, investigator, theory, and methodological triangulation, and offers insights into evaluating grounded theory. Additionally, the text addresses the importance of a structured approach to selecting qualitative research methods and ensuring the credibility and usefulness of research findings.
Managing Quality In Qualitative Research
- 1. Managing Quality in Qualitative Research
- 2. How to Address Research Quality Standards, Criteria, Checklists and Guidelines Concepts of Triangulation Triangulation of Qualitative and Quantitative Research Focusing on Process and Transparency
- 3. How to Address Research Quality
- 4. The issue of quality in qualitative research is located at the crossroads of internal needs and external challenges
- 5. • Researchers interest • Can I trust the results that I have created? • Are the methods that I have used in this situation correct? • If I work with others will our results be similar? • Funding Institution approach • How is one qualitative approach more better than another?
- 6. • Journal Editor / Conference Editor decisions • Can I trust the rigour of this piece of work? • How do I know that this is the best approach? • Readers questions • Can I trust what I’m reading? • How much detail did the researcher go into when they were creating this?
- 7. It has become a crucial issue with the establishment of qualitative research as a valid method and in competition with other forms of work
- 8. The Independent Samples t-test compares the means of two independent groups in order to determine whether there is statistical evidence that the associated population means are significantly different.
- 9. What is the qualitative equivalent?
- 11. • Evaluation built on Standardisation • One-Size-Fits-All Approach • Criteria Appropriate to Research or Research Appropriate to Criteria • How to Assess Research Quality in a Sensitive Way Different potential ways to approach this
- 12. The formulation of research ‘criteria’ is only one solution to the problem
- 13. Standards, Criteria, Checklists and Guidelines
- 14. Reformulating Traditional Criteria Aim here is to make the whole research process more transparent • Exact and Coherent Guidelines are used • Interviews are transcribed in a certain way using specific methods • Distinction between verbatim statements in field notes and summaries are given • Reliability of whole process can be seen in reflexive documentation
- 15. Reformulating Traditional Criteria Validating an Interview Situation • Are interviewees given any cause to construct a biased version of their experiences that doesn’t fit with their views? • Script is analysed to look for these specific points and to see whether the data is reliable Communication Validation • Participants can be involved in a secondary research study to assist in validation of data • Participants can reflect on previous answers and then better articulate their thoughts
- 16. Reformulating Traditional Criteria Procedural Validation • Validation is looked at for the entire research process and the different relationships that take place: • Relationship between what is being observed and the larger cultural, historical, and organisational contexts within which the observations are made • The relationship among the observer, the observed, and the setting • The perspectives that are used to render an interpretation of any ethnographic data • The role of the reader in the final product • The issue of author style to render the description or interpretation
- 17. Alternative, method-appropriate criteria Instead of using traditional metrics, we use those that are more credible for qualitative work • Are findings grounded in the data? • Is sampling appropriate • Are data weighed correctly? • Are inferences logical? • Analytic strategies applied correctly? • Alternative explanations accounted for? • Is the category structure appropriate? • Can decisions be justified? • What is the degree of researcher bias? • What strategies were used for increasing credibility?
- 18. Traditional criteria tend to miss the specific qualities of qualitative research. Alternative criteria mostly come without benchmarks for distinguishing good from bad.
- 19. Evaluating Grounded Theory Specific suggestions on evaluating Grounded Theory studies come from Charmaz (2006) who suggests breaking this down into four criteria • Credibility • Originality • Resonance • Usefulness
- 20. Evaluating Grounded Theory • Has the research achieved familiarity with the setting or topic? • Is the data sufficient to merit your claims? • Think about range, number and depth of observations contained in the data • Have you made systematic comparisons between observations and between categories • Do the categories cover a wide range of empirical observations • Are there strong links between the gathered data and your argument and analysis? • Has the research provided enough evidence for your claims to allow the reader to form an independent assessment, and agree with your claims? Credibility
- 21. Evaluating Grounded Theory • Are your categories fresh, do they offer new insights? • Does your analysis provide a new conceptual rendering of the data? • What is the social and theoretical significance of this work? • How does your theory challenge, extend, or refine current ideas, concepts and practices? Originality
- 22. Evaluating Grounded Theory • Do the categories portray the fullness of the studied experience? • Have you revealed both liminal and unstable taken-for-granted meanings • Have you drawn links between larger collectivities or institutions and individual lives? • Does your grounded theory make sense to your participants or people who share their circumstances? Does your analysis offer them deeper insights about their lives and world? Resonance
- 23. Evaluating Grounded Theory • Does your analysis offer interpretations that people can use in their everyday worlds? • Do your analytic categories suggest any generic processes? • If so, have you analysed these processes for implications? • Can the analysis spark further research in other substantive areas? • How does your work contribute to knowledge, how does it contribute to making a better world!? Usefulness
- 24. A Framework for Analysing Qualitative Research Data Research should be: • Contributory in advancing wider knowledge or understanding • Defensible in design by providing a research strategy • Rigorous in conduct through the systemic collection, analysis, and interpretation of qualitative data • Credible in Claim through offering well- founded and plausible arguments about the significance of the data generated Quality in Qualitative Evaluation: A framework for assessing research evidence A Quality Framework Liz Spencer, Jane Ritchie, Jane Lewis and Lucy DillonNational Centre for Social Research Government Chief Social Researcher’s Office 418771_CaboffQTY_SHORTV3_AW2 11/7/03 12:39 Page c3
- 25. Standards will only be helpful if they apply to qualitative research in general and not to specific approaches
- 27. Triangulation is the method of location of a point from two others of known distance apart, given the angles of the triangle formed by the three points x y° z°
- 28. Triangulation includes researchers taking different perspectives on an issue under study or more generally in answering research questions. These perspectives can be substantiated by using several methods or theoretical approaches, Triangulation should produce knowledge at different levels, which means they go beyond the knowledge made possible by one approach and thus contribute to promoting quality in research.
- 29. Types of Triangulation • Data Triangulation • The use of different sources of data • Allows the researcher to reach maximum profit when using the same methods • Investigator Triangulation • The use of different observers or interviewers • This is not the sharing of work, it is a systemic comparison of researchers influences on the issue • Theory Triangulation • Approaching the data with multiple perspectives and hypothesis in mind • Points are put side by side to assess their utility and power. • Methodological Triangulation • Different methods are used to look at the same issue
- 30. Within-Methods Triangulation Method Issue of ResearchApproach 1 Approach 2 Example of this technique is the episodic interview which combines questions and narratives.
- 31. Within-Methods Triangulation Interviews can be used to understand the everyday knowledge that people have Semi-Structured interviews can include narratives where participants talk at length about specific aspects that they have encountered BUT there is the chance for participants to switch from a narrative discussion to a descriptive, argumentative, or other non-native form of presentation Episodic Interviews
- 32. Within-Methods Triangulation Episodic Interviews Situation 1 Situation 3 Situation 2 Episodic Knowledge Episodic Interview Narrative Presentation Argumentative- Theoretical Presentation
- 33. Within-Methods Triangulation Episodic Interviews Situation 1 Situation 3 Situation 2 Episodic Knowledge Concept 1 Concept 2 Sub-Concept 1 Sub-Concept 3Sub-Concept 2 Episodic Interview Semantic Knowledge Narrative Presentation Argumentative- Theoretical Presentation
- 34. Within-Methods Triangulation Episodic Interviews Situation 1 Situation 3 Situation 2 Episodic Knowledge Concept 1 Concept 2 Sub-Concept 1 Sub-Concept 3Sub-Concept 2 Episodic Interview Semantic Knowledge Narrative Presentation Argumentative- Theoretical Presentation
- 35. Argumentations Subjective Definitions Examples Repisodes Narratives of Situations Stereotypes Situational Narratives based on different levels of concreteness Repisodes regularly occurring situations, no longer based on a clear local and temporal reference Examples abstracted from concrete situations, and metaphor also ranging to clichés and stereotypes Subjective Definitions which are asked for within sessions Linked to Argumentative Definitions of terms, explanations of concepts Within-Methods Triangulation
- 36. Between-Methods Triangulation Method 1 Issue of Research Choose at least one method which is specifically suited to exploring the structural aspects of the problem and at least one which can capture the essential elements of its meaning to those involved Method 2
- 37. Triangulation of Qualitative and Quantitative Research
- 38. In this module we’ve dealt exclusively with qualitative work. However, it is possible to combine qualitative and quantitative methods in order to increase research quality through triangulation
- 39. Quantitative Qualitative continuous collection of both sorts of data
- 40. Qualitative Quantitative continuous field research wave 1 wave 2 wave 3
- 43. Linking Qualitative and Quantitative Results Results may converge, that is, are consistent completely, in general, by tendency or partially. For example - answers in a representative survey may match with statements from semi-structured interviews Results may be complementary. Interviews can provide deeper, more detailed explanations to complement results from a questionnaire Results may diverge. For example, interviews may produce views that are different when compared to questionnaires. This would cause the need for further research.
- 44. Triangulation of qualitative and quantitative research is not per se a quality indicator for qualitative research, but it can contribute to overall quality
- 45. Qualitative Research Quantitative Research Data Sets Single Cases Triangulation Focus Groups Interviews Questionnaires Lab Studies
- 46. Positioning Triangulation in the Research Process Exploration Data Collection Data Interpretation Generalisation
- 47. Focusing on Process and Transparency
- 48. Quality in research is produced through the whole process.
- 49. Questions to ask for selecting a qualitative research method I. What do I know about the issue of my study or how detailed is my knowledge already? II. How developed is the theoretical or empirical knowledge in the literature about this issue? III. Is my interest in more generally exploring the field and the issue of my study? IV. What is the background of my study and which methods fit with this? V. What do I want to get close to in my study? I. Personal Experiences of a group of people / social process II. Reconstruction of underlying research structures
- 50. Questions to ask for selecting a qualitative research method VI. Do I start with a focused research question right away or do I start with an unfocused approach in order to develop the question? VII. Which aggregate do I want to study: personal experience, interactions or situations, or bigger entities like organisations or discourse? VIII.Is it more the single case I am interested in or the comparison of various cases? IX. Which resources (time, money, manpower, skills etc.) are available for my study?
- 51. Questions to ask for selecting a qualitative research method X. What are the characteristics of the field I want to study and of the people in it? What can you request of them and what not? XI. What is the claim of generalisation of my study? XII. What are the ethical issues to take into account that are affected by selecting a specific method?
- 52. Decide and reflect carefully whether you should use qualitative or quantitative research • Why qualitative research? • Which reasons do you have for the one or the other? • What are your expectations for the qualitative research that you are planning?
- 53. Reflect on the theoretical background of your knowledge interest • What is the impact of your setting on the research? • How open and closed is your access to what you want to study?
- 54. Plan your study carefully, but allow for reconsidering the steps and modifying according to the state of play • What are the resources available for the study? • How realistic are the aims of your research in relation to the available resources • What are necessary and appropriate shortcuts x y z a b
- 55. Plan your sampling carefully! • What are your cases? • What do they stand for?
- 56. Think about whom in the field you should contact and inform about your research. Reflect about the relation to establish to field subjects • What can you learn about your research field and issues from the way you get into the field or are rejected?
- 57. Think about why you chose your methods for collecting data • Is it a decision for a favourite method or for habitual reasons? • What could or would alternative methods provide? • What are the impacts of the methods you use on your data and your knowledge?
- 58. Plan carefully how to document your data and research experiences • How exactly should you write your notes? • What are the influences of the documentation on your research and on your field subjects? • What are the impacts of the documentation on your methods of collection and analysis?
- 59. Think about the way that you want to present what you have experienced in the field and found in your research • What are the target audiences of your writing? • What is it mainly you want to convince them about your research? • What is the impact of the format of your writing on your research and its findings?
- 60. Plan how to establish the quality of your research • What are the quality criteria your research should meet? • How should these criteria be realised? • What is their impact on your research and your field subjects or relationships?
- 61. How to Address Research Quality Standards, Criteria, Checklists and Guidelines Concepts of Triangulation Triangulation of Qualitative and Quantitative Research Focusing on Process and Transparency
- 62. Information in this presentation was based on…