Using Feedback Workshops in Qualitative Evaluation
- 1. Using Feedback Workshops in Qualitative Evaluation By: Brianne Labute For: Qualitative Research Methods EDRD 6000 University of Guelph March 3rd, 2015 This work is licensed under the Creative Commons Attribution 4.0 International License. For more information please visit http:// creativecommons.org/ licenses/by-nc-nd/4.0/
- 2. What is a feedback workshop? A feedback workshop is a meeting between evaluator(s) and stakeholders to review and discuss the evaluation findings Source: Brianne Labute
- 3. Who is considered a stakeholder? • Key program/project staff or volunteers, administrators, or beneficiaries • Could also be external stakeholders – not directly involved but have a valuable contribution – such as community leaders or elected officials Source: Ridder, D . , Mostert, E . , and Wolters, H . A. (eds.) (2005), Learning Together To Manage Together; Improving Participation in Water Management (Osnabrueck: University of Osnabrueck, USF) , p. 11.
- 4. When are feedback workshops used in evaluation? • Can be used with any type of evaluation design – especially useful in evaluations that use qualitative methods • Feedback workshops can be conducted throughout the evaluation progress (often the case in a more participatory evaluation design); or they can be done only in the final stages • A report by the Kellogg’s Foundation suggests that “early feedback and a consultative relationship with stakeholders and project staff leads to a greater willingness by staff to disclose important and sensitive information to evaluators” (W.K. Kellogg Foundation, 2004)
- 5. Why use a feedback workshop? • To ensure the evaluator(s) properly interpreted the data and the values of the stakeholders are properly represented in the findings • This is especially important when working with qualitative data due to its rich, thick description • To increase the understanding of the evaluation process among stakeholders and the value of the findings • To empower stakeholders through engagement • To improve the accuracy of the findings • To create discussion about how the findings can be disseminated to influence change
- 6. A personal experience Source: Brianne Labute I was working in northern Ghana on a tourism development project. When interviewing a group of women selling yams about their experience with tourists a women approached us and started yelling. She said “ you people come here with your notebooks and ask all these questions. Why! We never see changes, we need water and better houses! We are suffering!” We were able to briefly explain that we were not coming with any funding, rather, we were there to assist the existing tourism association. The woman seemed only slightly less agitated. Luckily, we had already planned a community workshop later in the week to share more about our work. We invited her and were glad when she accepted. At the workshop there were many questions and suggestions for us from the community members. Afterwards, the community had a much better understanding of what we were trying to achieve and our research was enriched by their feedback. A win-win situation!
- 7. • Invite stakeholders. If there are many, it is suggested you involve a manageable subset of stakeholder representatives • Although suggestions vary depending on the context , it is generally suggested there be no more than 15 participants at a workshop to allow for meaningful dialogue • Send a workshop agenda and the draft report to participants in advance • Prepare materials to present the findings and guide discussion such as a PowerPoint presentation or handouts Tips on how to conduct a successful feedback workshop: Preparation Special Note: In some international contexts these steps may not be very logical. Rather, you should consult your community contact and find out the best way to spread the word about the workshop and planned activities. This could be an announcement through the local mosque, sign board, etc.
- 8. Preparation Continued • Determine workshop roles. Who will chair, take notes, greet participants, etc. • Ensure you have a meeting space conducive for good dialogue • If logistically possible, gathering over food and drinks is a great way to create a comfortable space for those attending • Meeting face-to-face is highly recommended. If this is not possible, other methods could be used to collect feedback such as online surveys or video conference calls Source: Brianne Labute
- 9. During the workshop • Brief stakeholders on the evaluation work and the findings • Invite stakeholders to identify problems of ambiguity and fact in the findings • Clarify any misunderstandings • Invite participants to discuss the next steps (ie. how should the results of the evaluation be used to make improvements) • Discuss how future evaluations can be improved
- 10. After the workshop • Revise the evaluation report as needed • Send the revisions to stakeholders • Adjust plans for future evaluations (if applicable) Source: Brianne Labute TIP! It is always beneficial to reflect on how the workshop could be better conducted next time. The workshop team should keep a notebook handy to record any ideas.
- 11. What are some benefits to using feedback workshops? • Involving stakeholders helps ensure a transparent process • It produces more reliable and accurate findings Increased ownership of the recommendations among stakeholders • Action on recommendations is more likely to be taken • Provides opportunity to build capacity among participants
- 12. What are some challenges when using feedback workshops? • Feedback processes can be time consuming • Feedback can be “based on facts, observation, and past knowledge; or almost entirely subjective, based on emotional reactions, defensiveness, loyalty (to family, friends, or class), cultural gaps, or distrust of outsiders” (University of Kansas, 2014b). • There is potential for conflict – Stakeholders may want interpret findings to their benefit
- 13. References Gullickson, R., Stufflebeam, D. (2001). Feedback Workshop Checklist. Retrieved from http://www.wmich.edu/evalctr/archive_checklists/feedbackworkshop.pdf Organization for Economic Co-operation and Development. (2001). Evaluation Feedback and Effective Learning and Accountability. Retrieved from file:///Users/briannelabute/Desktop/Feedback%20workshops/Evaluation%20Feedback%20for %20Effective%20Learning%20and%20Accountability.pdf The World Bank Institute. (2013). The Art of Knowledge Exchange: A Results Focused Planning Guide for Development Practitioners. Retrieved from http://wbi.worldbank.org/wbi/Data/wbi/wbicms/files/drupal-acquia/wbi/KEfull.pdf University of Kansas, Work Group for Community Health and Development. (2014a). Community Toolbox: Participatory Evaluation. Retrieved from http://ctb.ku.edu/en/table-of-contents/evaluate/evaluation/participatory-evaluation/main University of Kansas, Work Group for Community Health and Development. (2014b). Community Toolbox: Obtaining and Using Feedback from Participants. Retrieved from http://ctb.ku.edu/en/table-of-contents/maintain/maintain-quality-performance/feedback-from- clients/main W.K. Kellogg Foundation. (2004). Evaluation Handbook. Retrieved from https://www.wkkf.org/resource-directory/resource/2010/w-k-kellogg-foundation-evaluation- handbook