How to navigate the maze of evaluation methods and approaches
Download as PPTX, PDF2 likes2,686 views
This document discusses the overwhelming number of program monitoring and evaluation tools and approaches and provides steps to navigate this complexity. It introduces the BetterEvaluation podcast which aims to help navigate the many evaluation methods. It promotes focusing on logic models and outcome mapping. It also shares frameworks like the Rainbow Framework which maps over 200 methods across evaluation tasks, and lists competencies and standards for evaluation like the Program Evaluation Standards. The overall goal is to help people understand and select the most appropriate evaluation approaches.
How to navigate the maze of evaluation methods and approaches
- 1. Sharing information to improve evaluation
- 2. How to navigate the maze of evaluation methods and approaches A BetterEvaluation Podcast BetterEvaluation.org Simon Hearn & Ron Mackay
- 3. The Challenge: Overwhelming number and diversity of PME tools
- 4. Wikipedia page for Evaluation Methods
- 5. Steps Taken: Tools are just too many, focus on Log Frame and Outcome Mapping
- 6. Image: Michael Heilemann. http://www.flickr.com/photos/heilemann
- 8. Image: Lottery Monkey. http://www.flickr.com/photos/lotterymonkey/
- 9. Image: Simon Kneebone. http://simonkneebone.com/
- 10. Image: Susan Kistler. http://aea365.org
- 14. Simple organisational cycle Plan Assess Act
- 15. CIPP Model Context Inputs Process Product
- 17. Evaluation is the activity of systematically collecting, analyzing and reporting information so that it can be used by appropriate people to answer their questions so that they can make decisions about the future of a project or program.
- 18. • Who carries out the ‘activity’? • Who or what is an evaluator? • How does an activity become ‘systematic’? • What activites and skills do ‘collecting, analysing and reporting’ involve? • How do you go about ‘collecting, analysing and reporting’? • Is that an exhaustive list of ‘the activity’ that is called ‘evaluation’ • What kind of ‘information’ is being referred to? • What is the thing or things you collect information about? • Who should be involved in an evaluation? • Who are the ‘appropriate people’ who ‘use’ the ‘information’ supplied by an evaluation? • And for what purposes can they ‘use’ this information? • What kinds of questions can they ask?
- 20. Rainbow Framework Over 200 methods/options related to 35 tasks in 7 clusters
- 28. Rainbow Framework Over 200 methods/options related to 35 tasks in 7 clusters
- 31. See IOCE.net for a list of national evaluation societies
- 32. Evaluation Competencies 1. Reflective Practice 2. Technical Practice 3. Situational Practice 4. Management Practice 5. Interpersonal Practice Competencies for Canadian Evaluation Practice, Canadian Evaluation Society
- 33. Evaluation Standards 1. Utility Standard 2. Feasibility Standards 3. Propriety Standards 4. Accuracy Standards 5. Evaluation Accountability Standards The Program Evaluation Standards developed by the JCSEE
- 34. How to navigate the maze of evaluation methods and approaches A BetterEvaluation Podcast BetterEvaluation.org Simon Hearn & Ron Mackay
Editor's Notes
- #4: There is increasing recognition of the importance of evaluation. But when NGOs or government agencies start to plan an evaluation, or to engage an evaluator, it is difficult to find accessible, comprehensive and credible information about how to choose the right combination of evaluation methods, and how to implement them well – or how to manage these issues with a potential evaluator. Many guides to evaluation focus on only a few types of data collection methods and a few types of research designs. Few guides cover newer types of data collection (such as GIS logging on mobile phones, or tracking social media mentions), designs and strategies that can be used when experimental or quasi-experimental research designs are not feasible or appropriate, and methods to address issues such as clarifying and negotiating the values that should underpin the evaluation. There is information about some of these available on various web sites, but they can be difficult to locate, of varying quality, and not co-ordinated. And much of the information is partisan for particular methods, or does not include examples from practice.
- #5: There is increasing recognition of the importance of evaluation. But when NGOs or government agencies start to plan an evaluation, or to engage an evaluator, it is difficult to find accessible, comprehensive and credible information about how to choose the right combination of evaluation methods, and how to implement them well – or how to manage these issues with a potential evaluator. Many guides to evaluation focus on only a few types of data collection methods and a few types of research designs. Few guides cover newer types of data collection (such as GIS logging on mobile phones, or tracking social media mentions), designs and strategies that can be used when experimental or quasi-experimental research designs are not feasible or appropriate, and methods to address issues such as clarifying and negotiating the values that should underpin the evaluation. There is information about some of these available on various web sites, but they can be difficult to locate, of varying quality, and not co-ordinated. And much of the information is partisan for particular methods, or does not include examples from practice.
- #6: There is increasing recognition of the importance of evaluation. But when NGOs or government agencies start to plan an evaluation, or to engage an evaluator, it is difficult to find accessible, comprehensive and credible information about how to choose the right combination of evaluation methods, and how to implement them well – or how to manage these issues with a potential evaluator. Many guides to evaluation focus on only a few types of data collection methods and a few types of research designs. Few guides cover newer types of data collection (such as GIS logging on mobile phones, or tracking social media mentions), designs and strategies that can be used when experimental or quasi-experimental research designs are not feasible or appropriate, and methods to address issues such as clarifying and negotiating the values that should underpin the evaluation. There is information about some of these available on various web sites, but they can be difficult to locate, of varying quality, and not co-ordinated. And much of the information is partisan for particular methods, or does not include examples from practice.