Learning_Unit_3
- 1. SSR 3014 Project Evaluation and Cost Benefit Analysis Learning Unit 3: Introduction to Monitoring and Evaluation
- 2. Introduction Project Development Lifecycle • Define • Plan • Monitor • Evaluate
- 3. Introduction WHY DO WE NEED MONITORING AND EVALUATION?
- 4. Basic Principle • What is Monitoring? • What is Evaluation? • Why is it IMPORTANT? • What is involved?
- 5. Monitoring • Systematic collection and analysis of information. • Improve efficiency and effectiveness. • Ensure that project is on track. • Measures progress towards objectives. • Identifies problems.
- 6. Evaluation • Assessing whether a project is achieving its intended objectives. • Conducted periodically. • Internal or external. • Focus on outcomes and impacts.
- 7. Commonality between Monitoring and Evaluation • Geared towards learning and focused on: 1. Efficiency 2. Effectiveness 3. Outcomes and impacts
- 8. Why do Monitoring and Evaluation? • Review progress • Identify problems in planning or implementation • Make adjustment to enhance impact
- 10. Why is Monitoring and Evaluation important? • Tracking resources • Feedback on progress • Improving project effectiveness • Informing decisions • Promoting accountability • Demonstrating impact • Identifying lesson learned
- 11. Indicators • Outcome vs process indicators • Qualitative Vs quantitative
- 12. Developing Indicators 1. Identify the problem 2. Decide what result should look like (impact indicator) 3. Decide how the result should be achieved (process indicator) 4. Develop indicator for effectiveness 5. Develop indicator for efficiency
- 13. Monitoring Tools • Progress report • Regular meeting • Budget and income expenditure account • Structured survey or one-on-one interview • Systematic review of official statistics • Gantt chart or Critical Path Analysis chart • Process approach (fishbone diagram)
- 14. EVALUATION
- 15. EVALUATION • WHAT ARE THEY? • WHY WE NEED THEM? • WHEN DO WE USE THEM?
- 16. EVALUATION A process to determine the value or worth of the object of interest (e.g. project, program) against a standard of acceptability Begins when project goals and objectives are being developed Done by collaborative effort of stakeholders Results must be relevant to stake holders to be used effectively Driving force for planning, improving and demonstrating
- 17. Can be a political process 1. Negative result may lead to reduction on funding 3. Reporting fairly but may comply with governments regulation 2. Results intentionally skewed by reporting success only EVALUATION
- 18. FUNCTION To answer these question How well did we do? How much did we do?
- 19. WHY EVALUATE? Improve program design and implementation • It is important to periodically assess and adapt your activities to ensure they are as effective as they can be. Evaluation can help you identify areas for improvement and ultimately help you realize your goals more efficiently. Demonstrate program impact • Evaluation enables you to demonstrate your program’s success or progress. The information you collect allows you to better communicate your program's impact to others, which is critical for public relations, staff morale, and attracting and retaining support from current and potential funders
- 20. WHY EVALUATE? • Determine program outcomes • Identify program strengths • Identify and improve weaknesses • Justify use of resources • Increased emphasis on accountability • Professional responsibility to show effectiveness of program
- 21. Project Evaluation Purposeful, systematic, and careful collection and analysis of information used for the purpose of documenting the effectiveness and impact of programs, establishing accountability, and identifying areas needing change and improvement
- 22. What evaluation does? Looks at the results of your investment of time, expertise, and energy, and compares those results with what you said you wanted to achieve
- 23. Types of Evaluation • Outcome • Implementation • Formative • Summative
- 24. Outcome evaluation • What: Identifies the results or effects of a program • When: You want to measure students’ or clients’ knowledge, attitudes, and behaviors as a result of a program • Examples: Did program increase achievement, reduce truancy, create better decision-making? Implementation evaluation • What: Documents what the program is and to what extent it has been implemented • When: A new program is being introduced; identifies and defines the program; identifies what you are actually evaluating • Examples: Who receives program, where is program operating; is it being implemented the same way at each site?
- 25. Formative evaluation • conducted during program development and implementation and are useful if you want direction on how to best achieve your goals or improve your program Summative evaluation • should be completed once your programs are well established and will tell you to what extent the program is achieving its goals.
- 26. FORMATIVE EVALUATION •Determines who needs the program, how great the need is, and what can be done to best meet the need. •needs assessment can help determine what audiences are not currently served by programs and provide insight into what characteristics new programs should have to meet these audiences’ needs. Need assessment •Examines the process of implementing the program and determines whether the program is operating as planned •Can be done continuously or as a one-time assessment • Results are used to improve the program •A process evaluation may focus on the number and type of participants reached and/or determining how satisfied these individuals are with the program. Process/ Implementation of evaluation
- 27. SUMMATIVE EVALUATION • Investigates to what extent the program is achieving its outcomes. • These outcomes are the short-term and medium-term changes in program participants that result directly from the program Outcome evaluation • Determines any broader, longer-term changes that have occurred as a result of the program. • These impacts are the net effects, typically on the entire school, community, organization, society, or environment Impact evaluation
- 30. Tools for Evaluating Surveys Forced choice or open-ended responses Interviews Structured (fixed questions) or in-depth (free flowing) Focus groups Like interviews but with group interaction Observations Actually monitor and evaluate behavior
- 31. Comparing surveys and observation Survey • Efficient • Accuracy depends on subject’s honesty • Difficult to develop reliable and valid survey • Low response rate threatens reliability, validity, & interpretation Observation • Time & labor intensive • Inter-rater reliability must be established • Captures behavior that subjects unlikely to report • Useful for observable behavior
- 32. Appropriateness of Interviews Use interviews to answer these questions: What does program look and feel like? What do stakeholders know about the project? What are stakeholders’ and participants’ expectations? What features are most salient? What changes do participants perceive in themselves?
- 34. Plan for evaluation • Plan while you are designing your program • Consider ho you will share your findings. • Review project goal and objectives • Determine and engage with stakeholders • Develop and decide evaluation design/instrument • Develop time line Data collection • Must have an idea about your target audience and how accessible they are to you as well. • Survey, records, telephone interviews. • Plan and administer a pilot test • Review result to refine instrument or collection procedure • Set schedule Evaluation Process
- 35. Analyze data • Process data • Analyze data • Interpret • Who • How • What are the limitations Report Findings • Demonstrate the effectiveness of your project, identify ways to improve future projects, modify project planning, demonstrate accountability, and justify funding • Stakeholders need • Audience • Key messages • Format Evaluation Process