Program Evaluation: Forms and Approaches by Helen A. Casimiro
Download as PPT, PDF6 likes4,852 views
This document discusses different forms and approaches to program evaluation. It describes five forms of evaluation: 1) Proactive Evaluation which occurs before program design to synthesize knowledge for decisions, 2) Clarificative Evaluation which occurs early in a program to document essential dimensions, 3) Participatory/Interactive Evaluation which occurs during delivery to involve stakeholders, 4) Monitoring Evaluation which occurs over the life of an established program to check progress, and 5) Impact Evaluation which assesses the effects of a settled program. It also outlines several evaluation approaches including behavioral objectives, four-level training outcomes, responsive, goal-free, and utilization-focused evaluations.
Program Evaluation: Forms and Approaches by Helen A. Casimiro
- 2. Program Evaluation A systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency.
- 3. Evaluation can be classified into five categories or Forms PROACTIVE EVALUATION (FORM A) takes place before a program is designed. The purpose is to synthesize knowledge for decisions about how to best develop a program in advance of its planning and implementation.
- 4. CLARIFICATIVE EVALUATION (Form B) which takes place early in the delivery of a program. The purpose is to provide knowledge that identifies and documents the essential dimensions of a program to make them explicit to stakeholders.
- 5. PARTICIPATORY/INTERACTIVE EVALUATION (Form C) which takes place during the delivery of a program. The purpose is to provide knowledge for decisions related to continuous improvement by involving program providers in the evaluation process.
- 6. MONITORING EVALUATION (Form D) which takes place over the life of a program that is well established and ongoing. The purpose is to provide knowledge to check that the program is “on track” and to provide a basis for its refinement.
- 7. IMPACT EVALUATION which is used to assess the impact of a settled program. The purpose is to determine the effects of the program in terms of the criteria selected to judge its success.
- 8. EVALUATION MODELS BEHAVIORAL OBJECTIVES APPROACH This approach focuses on the degree to which the objectives of a program, product, or process have been achieved. The major question guiding this kind of evaluation is, “Is the program, product, or process achieving its objectives?”
- 9. THE FOUR-LEVEL MODEL. This approach is most often used to evaluate training and development programs (Kirkpatrick, 1994). It focuses on four levels of training outcomes: reactions, learning, behavior, and results. The major question guiding this kind of evaluation is, “What impact did the training have on participants in terms of their reactions, learning, behavior, and organizational results?”
- 10. EVALUATION APPROACHES are conceptually distinct ways of thinking about, designing, and conducting evaluation efforts
- 11. Responsive Evaluation This approach calls for evaluators to be responsive to the information needs of various audiences or stakeholders. The major question guiding this kind of evaluation is, “What does the program look like to different people?” Goal-Free Evaluation This approach focuses on the actual outcomes rather than the intended outcomes of a program. Thus, the evaluator has minimal contact with the program managers and staff and is unaware of the program’s stated goals and objectives. The major question addressed in this kind of evaluation is, “What are all the effects of the program, including any side effects?”
- 12. Adversary/Judicial Approaches These approaches adapt the legal paradigm to program evaluation. Thus, two teams of evaluators representing two views of the program’s effects argue their cases based on the evidence (data) collected. Then, a judge or a panel of judges decides which side has made a better case and makes a ruling. The question this type of evaluation addresses is, “What are the arguments for and against the program?” Consumer-Oriented Approaches The emphasis of this approach is to help consumers choose among competing programs or products. Consumer Reports provides an example of this type of evaluation. The major question addressed by this evaluation is, “Would an educated consumer choose this program or product?”
- 13. Expertise/Accreditation Approaches The accreditation model relies on expert opinion to determine the quality of programs. The purpose is to provide professional judgments of quality. The question addressed in this kind of evaluation is, “How would professionals rate this program?” Utilization-Focused Evaluation According to Patton (1997), “utilization-focused program evaluation is evaluation done for and with specific, intended primary users for specific, intended uses”. As such, it assumes that stakeholders will have a high degree of involvement in many, if not all, phases of the evaluation. The major question being addressed is, “What are the information needs of stakeholders, and how will they use the findings?”
- 14. Participatory/Collaborative Evaluation The emphasis of participatory/collaborative forms of evaluation is engaging stakeholders in the evaluation process, so they may better understand evaluation and the program being evaluated and ultimately use the evaluation findings for decision-making purposes. As with utilizationfocused evaluation, the major focusing question is, “What are the information needs of those closest to the program?” Empowerment Evaluation This approach, as defined by Fetterman (2001), is the “use of evaluation concepts, techniques, and findings to foster improvement and self-determination” (p. 3). The major question characterizing this approach is, “What are the information needs to foster improvement and selfdetermination?”
- 15. Organizational Learning Some evaluators envision evaluation as a catalyst for learning in the workplace (Preskill & Torres, 1999). Thus, evaluation can be viewed as a social activity in which evaluation issues are constructed by and acted on by organization members. The major question in this case is, “What are the information and learning needs of individuals, teams, and the organization in general?” Theory-Driven Evaluation This approach to evaluation focuses on theoretical rather than methodological issues. The basic idea is to use the “program’s rationale or theory as the basis of an evaluation to understand the program’s development and impact” (Smith, 1994, p. 83). The major focusing questions here are, “How is the program supposed to work? What are the assumptions underlying the program’s development and implementation?”
- 16. Success Case Method This approach to evaluation focuses on the practicalities of defining successful outcomes and success cases (Brinkerhoff, 2003) and uses some of the processes from theory-driven evaluation to determine the linkages, which may take the form of a logic model, an impact model, or a results map. Evaluators using this approach gather stories within the organization to determine what is happening and what is being achieved. The major question this approach asks is, “What is really happening?”
- 17. Using the Forms in Practical Settings
- 18. Using the Forms in Practical Settings (cont.)…
- 40. Clarificative
- 41. Interactive
- 42. Monitoring
- 43. Impact
- 46. THANK YOU!!