Using qualitative data to tell your story of change on campus
- 1. Using Qualitative Data to Tell Your Story of Change on Campus Evaluating the Dinner Basket Conversations
- 2. One Initiative: Two Research Projects The Initiative: What’s a Dinner Basket Conversation? Dinner Basket Conversations (DBC) are a means by which groups of people can gather together to create and eat a meal, while engaging in conversation around a chosen topic. Evidence suggests that more ease and flow is created in conversation when it is held in the context of sharing a meal. The purpose is to gather information about the “culture of substance use” and peripherally influence cultural shifts. In the case of this study, the Dinner Basket Conversations that the stories were gleaned from focused on substance use and relationships with substances.
- 3. Facilitators volunteer to host a DBC. They invite a group of people they would like to engage in the conversation and purchase the groceries for a meal, usually provided by the coordinator of the DBC project. The group prepares the meal together. Facilitators are given a suggested list of conversation starters.
- 4. . They are encouraged to make notes on the themes and discussions that arise and, with permission from the participants, take pictures of the event. An evaluation of the event is handed out to the participants after it is over and these evaluations are collected and given to the coordinator.
- 5. Most Significant Change Stories • In brief: the most significant change (MSC) technique is a means of “monitoring without indicators” (but can also be used in evaluations) and in this case I adapted the technique to a research project.
- 6. MSC is a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved in both deciding the sorts of changes to be recorded and analysing the data collected. It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program. It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole.
- 7. The process involves the collection of significant change (SC) stories originating from the field level, and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff.
- 8. Once change stories have been captured, selected groups of people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of these reported changes, ultimately deciding which is most significant. In large programs there may be multiple levels at which SC stories are pooled and then selected. When the technique is implemented successfully, whole teams of people begin to focus their attention on program impact.
- 9. Research Question • Are Dinner Basket Conversations a useful way of changing the culture of substance use among Selkirk College Students?
- 10. Significance of the Study • Given the difficulty in measuring abstract concepts such as changes in culture, there is value in measuring or understanding cultural shifts by listening carefully to the impact and significance of stories told by those embedded in the current culture. Subtle shifts in view and understanding occur as we engage communities in conversations. Sometimes we find that a more dramatic shift in perspective occurs as a result of one significant story.
- 11. This research makes use of an evaluative technique called “Most Significant Change Stories” to tease out the story that had the most impact on the participants and the facilitators during the Dinner Basket Conversations. I sought to analyse the impact of this story to further inform the facilitators of future Dinner Basket Conversations. I also assigned meaning to the story in terms of it potentially being an example of culture change in regards to substance use.
- 12. Theoretical Perspective • The theoretical perspective has its roots in Systems Theory, but more specifically as it applies to Adaptive Systems Theory. Since we are continually responding to environmental changes, Adaptive Systems Theory recognizes that cultural change occurs as a result of feedback loops that allow response to changes, events and stories.
- 13. MSC is most useful: Where it is not possible to predict in any detail or with any certainty what the outcome will be • Where outcomes will vary widely across beneficiaries • Where there may not yet be agreements between stakeholders on what outcomes are the most important • Where interventions are expected to be highly participatory, including any forms of monitoring and evaluation of the results. http://mande.co.uk/special-issues/ most-significant-change-msc/
- 14. Limitations of the Study • A relatively small sample of “significant stories” were gathered despite over 30 requests that were sent out to facilitators. Nine people responded with stories that they deemed significant and reported that these in particular impacted their perception and caused a shift in thinking. • The research method is very far to the left on the continuum of qualitative research methods and was not primarily designed as a research tool, but rather an evaluative tool. Evaluating stories in regards to their impact is a highly subjective endeavour and recognized as such.
- 15. Research Methodology: Research Questions • Tell me how you learned about Dinner Basket Conversations? • b. Please take a few minutes to reflect on the culture changes that may have happened as a result of the Dinner Basket Conversations? • c. What has been the most significant culture change for you associated with your participation in the Dinner Basket Conversations? • How have your ideas, behaviours and attitudes about substance use and people who use substances changed since participating in Dinner Basket Conversations? • e. From your point of view, describe a story that best illustrates the most significant change in the culture of substance use that you have experienced or heard about as a result of Dinner Basket Conversations being offered at Selkirk College. • f. Why is this story significant to you?
- 16. Population: Facilitators of DBC’s • Note: the facilitators were chosen to share their most significant stories rather than the participants because they had been exposed to a number of DBC rather than just one, and had a larger pool of stories and experiences to draw from. Participants were most often students who have since left the institution and are no longer available.
- 17. Data Collection Method • a. Two third-year Student Nurse research assistants emailed former facilitators of DBC describing the purpose of the research and asking if they would be willing to take part in a half-hour interview. • b. Respondents were provided with an Informed Consent that included a description of the study, its purpose and goals, and a statement of confidentiality. • c. After signing the consent, nine facilitators were interviewed and their stories were transcribed. • d. The stories were then vetted through a student panel and two were chosen. The remaining two were vetted through a panel of Selkirk College management staff and one story was chosen as the most significant.
- 18. Research Findings The Most Significant Change Story was entitled, The Sports Team Story
- 19. Story
- 20. Impact • The impact of this story on the facilitator was profound. She reported feeling quite touched by the willingness of the young man to make himself vulnerable and by the responses from his teammates. She reported that she thought, in the future, if this young man found himself in a similar situation he would probably seek out friends to either talk to or accompany him, thus perhaps mitigating risk. She also thought that his teammates were more dialed in to the possibility that their teammate might be struggling and that they might need to keep an eye on him. • She wondered if this may be the beginning of a small culture shift where the team might feel a responsibility to look out for one another at the social/emotional level.
- 21. Conclusions and Recommendations •Although “The Sports Team Story” was chosen as the most significant, it was representative of many other stories told by facilitators that exemplified the fact that cooking and eating together while having a conversation about difficult topics can be an important tool to loosen up the reluctance to share personal stories and honestly address troublesome issues.
- 22. As opposed to the “classroom” or “focus group” format, cooking a meal together involves teamwork, fun, and bonding that is not related to the tough topics of discussion. By the time the meal is served, the participants have gotten to know one another, had some fun laughing and teasing one another, and have relaxed into the environment
- 23. This sets the stage for a more meaningful discussion, and the stories that have been shared indicate that people really want to discuss this topic in a facilitated way - but not in an overly formalized way. Some students have mentioned that they cannot just talk to their friends about drug and alcohol use because everyone is too busy trying to be “cool”.
- 24. It is recommended that the Dinner Basket Conversations continue as they appear to not only gather important information for the college to use as we move forward with Healthy Campus initiatives, but also have an ameliorating effect on attitudes and behaviours that may not be serving the participants well. It is anticipated that this will lead to culture change as more and more participants become increasingly aware of the impact of unhealthy and healthy relationships with substances.
- 25. Thematic Analysis • Thematic analysis is one of the most common forms of analysis in qualitative research. It emphasizes pinpointing, examining, and recording patterns within data. Themes are patterns across data sets that are important to the description of a phenomenon and are associated to a specific research question.
- 26. . The themes become the categories for analysis. Thematic analysis is performed through the process of coding to create established, meaningful patterns. These phases are: familiarization with data, generating initial codes, searching for themes among codes, reviewing themes, defining and naming themes, and producing the final report.
- 27. Thematic Analysis is being used to evaluate the efficacy of the Dinner Basket Conversation Approach this year. The data is not all in yet but both the thematic observations from the DBC hosts and the participants feedback will inform this project. The data will be then be analyzed against the research question. "In what ways do Dinner Basket Conversations deepen students and staffs understanding of and response to sexualized violence, issues of consent and rape culture?"
- 28. Why Two Different Methodologies? • The MSCS methodology was used to evaluate the DBC’s about Healthy Relationships with Substances. Our goal in these DBC’s was to give students an opportunity to talk about their own substance use and that of their peer group. We were hopeful that because of the discussions some culture change might occur. The stories would hopefully reflect this. • We are using the Thematic Analysis for the DBC about Healthy Sexuality and Sexual Violence on Campus because we are measuring the efficacy of the tool, DBC, not measuring culture change.