Use of evaluation findings; types and influences
1 like1,647 views
The document discusses the types of evaluation use, including instrumental, conceptual, and symbolic, while highlighting common issues such as misuse and non-use. It emphasizes the importance of quality, context, and stakeholder involvement in influencing evaluation use, noting that use can often be unintended and unpredictable. The conclusions stress that engagement between evaluation clients and evaluators is essential for effective use, and organizations can significantly affect how evaluations are utilized.
Use of evaluation findings; types and influences
- 1. Use of evaluation findings: Types and influences National Health Promotion Conference Neuchâtel Switzerland 19 January 2017 Dr Glenn O’Neil oneil@owlre.com www.owlre.com
- 2. 2 Types of use Use Instrumental ConceptualSymbolic Process Misuse Non-use = most common types of use found
- 3. 3 What influences use? Quality Influences Credibility Relevance Comm. quality Timeliness Findings Decision - making Info. needs Political climate Competing Info. Receptiveness Personal characteristics ContextStakeholder involvement Internal context Organisational structure, culture and strategy = stronger influences on use
- 4. 4 “Engagement, interaction, and communication between evaluation clients and evaluators is critical to the meaningful use of evaluations” Johnson et al (2009) What influences use?
- 5. 5 – Assumption: what is used from an evaluation are the recommendations What is used from an evaluation? – Yet in a study by this author, out of 28 instances of use identified in two evaluations, 18 were not from recommendations – For example: Staff improve evaluation skills Modify programme planning Input into broad policy
- 6. 6 How does evaluation use occur? Individual Interpersonal Collective Informal change Formal change Four Years
- 7. 7 How does evaluation use occur? Use occurred: Unintended and direct Use occurred : Unintended and indirect Use occurred : Intended and direct Use occurred: Intended and indirect Use did not occur: Intended and indirect Use did not occur: Intended and direct More Less
- 8. 8 1. Use may not be instrumental and direct as expected 2. Stakeholder involvement is critical to use 3. Organisations can influence evaluation use 4. Use can be unpredictable, opportunistic & unexpected Conclusions
- 10. Main references • Ciarlo, J. A. (Ed.). (1981). Utilizing evaluation: Concepts and measurement techniques. Beverly Hills, CA: Sage Publications. • Contandriopoulos, D., & Brousselle, A. (2012). Evaluation models and evaluation use. Evaluation, 18(1), 61-77. • Cousins, J. B., & Leithwood, K. A. (1986). Current empirical research in evaluation utilization. Review of Educational Research, (536), 331-364. • Højlund, S. (2014a). Evaluation use in the organizational context–changing focus to improve theory. Evaluation, 20(1), 26-43. • Højlund, S. (2014b). Evaluation use in evaluation systems–the case of the European Commission. Evaluation, 20(4), 428-446. • Johnson, K., Greenseid, L. O., Toal, S. A., King, J. A., Lawrenz, F., & Volkov, B. (2009). Research on Evaluation Use A Review of the Empirical Literature From 1986 to 2005. American Journal of Evaluation, 30(3), 377-410. • Loud, M. L., & Mayne, J. (Eds.). (2013). Enhancing evaluation use: insights from internal evaluation units. Beverly Hills, CA: Sage Publications. • Mark, M. M., & Henry, G. T. (2004). The mechanisms and outcomes of evaluation influence. Evaluation, 10(1), 35- 57. • O’Neil, G. (2015) Communication evaluation in international organisations: methodology, influence and use. PhD thesis, The London School of Economics and Political Science (LSE). 10