Unit 6 Project Planning and Evaluation.ppt
- 1. Project Planning and Health Programmes Evaluation 1
- 2. Definitions • Project management: is the practice of initiating, planning, executing, controlling, and closing the work of a team to achieve specific goals and meet specific success criteria at the specified time • Project planning: is a procedural step in project management, where required documentation is created to ensure successful project completion. The project plan clearly defines how the project is executed, monitored, controlled and closed. 2
- 3. Definitions cont. Project planning requires an in-depth analysis and structuring of the following activities: 1.Setting project goals 2.Identifying project deliverables: desired outcomes/ desired change 3.Creating project schedules: objectives and activities 4.Creating supporting plans 3
- 4. 4 Description of Creation In the beginning God created the heaven and the earth. And the earth was unformed and void and darkness was on the face of the deep An God said ,”Let there be light.” and there was light And God saw the light, that it was good And God divided the light from the darkness Baseline data Further Programming activities Implementation of the programme Evaluation of the programme
- 5. Unit learning outcomes At successful completion of the unit, the students will be able to: 1.Design a community project proposal with application of the stages of the project planning process 2.To demonstrate understanding of community project implementation and evaluation. 5
- 6. 6 The Planning Spiral Situational Analysis (baseline) Who? Why? What? When? How? Define problem Priority Setting Option Appraisal Programming Implementation Evaluation Situational Analysis (2) Objectives Specific, Measurable, Achievable, Relevant, Time related Monitoring
- 7. Stages of the life cycle of a project Stage 1: Initiating the project Stage 2: Planning the project Stage 3: Designing the processes and outputs (deliverables) Stage 4: Implementing and tracking the project Stage 5: Evaluating and closing out the project 7
- 8. The five stages of the life cycle of a project Stage 1: Initiating the project •An effective way to get buy-in for a project or idea is to link it to what is important to the person or group you are approaching and demonstrate that you are openly soliciting their input. By doing so, they can help shape the concept. Stage 2: Planning the project •Assuming the project concept and feasibility have been determined, the plan-do-check-act (PDCA) cycle (see figure below) is directly applicable to project planning and management. 8
- 9. Stage 3: Designing the processes and outputs (deliverables) • When the project is approved, the project team may proceed with the content design along with the persons or items needed to implement the project. • The design process includes defining: • Measurements • The monitoring method • Status reporting protocols • Evaluation criteria • Design of the ultimate processes and outputs • Implementation schedules 9
- 10. Stage 4: Implementing and tracking the project • The project design team may also implement the project, possibly with the help of additional personnel. A trial or test implementation may be used to check out the project design and outputs to determine if they meet the project objectives. • Using the planned reporting methods, the implementation team monitors the project and reports on its status to appropriate interested parties at designated project milestones. Interim results may also be communicated to interested parties. The implementation team makes any course corrections and trade-offs that may be necessary and are approved. 10
- 11. Stage 5: Evaluating and closing out the project • Official closure of the project when the scheduled tasks have been completed. • Usually evaluations are done to determine: • Objectives met versus objectives planned • Actual tasks and events scheduled versus planned • Resources used versus planned resource usage • Costs versus budget • Organizational outcomes achieved versus planned outcomes; any unplanned outcomes • Effectiveness of project planning team (optional) • Effectiveness of implementation team (optional) • Team’s compilation of project documents, evaluations, and lessons learned • The project is then officially closed out. 11
- 12. 12
- 13. 13 What is Evaluation? • “Looking back at our work to see if we have reached our goals, and if not, examining where the problems lie” – “Was it worth doing?” • “Learning from experience” • “Constructive criticism” • “Systematic investigation of the merit, worth or significance of an objective” - CDC
- 14. Purpose • Managerial process to: – Improve health programmes – Improve health infrastructure – Guide allocation of resources • Decision making tool • Gain insight – Assess needs, desires, and assets of community members. – Identify barriers and facilitators to service use. 14
- 15. Purpose • Change practice • Assess effects • Efficient – doing it right at reasonable cost • Effective – doing the right thing/producing the desired result • Relevant • Acceptable • Affect participants 15
- 16. Purpose • Requires open mind • “Constructive Criticism” – Open communication – Critical analysis 16
- 17. Purpose • Judgment / assigning value to programme activities – Careful assessment & – Critical appraisal – Merit (quality) – Worth (cost-effectiveness) – Significance (importance) • Proposals for future action – Based on valid, reliable & sensitive information 17
- 18. Constraints of Evaluation • Predetermined, quantified objectives • Intricate relationship between health, social & economic sectors • Requires sensitive measures • Resistance to accept evaluation 18
- 19. Responsibility for Evaluation • Integral part of overall management • Individuals & groups responsible for development & application of that process or service Done by • Outsiders – impartial • Service providers – biased • Users of the service 19
- 20. 20 1. Health System Approach INPUTS PROCESS OUTPUT OUTCOME IMPACT FEEDBACK
- 21. 21 1.1 Inputs • Resources –Staff –Equipment –Money –Buildings • “Man, Machines, Money, Mortar”
- 22. 22 1.2 Processes • Affordability • Accessibility • Acceptability • Availability • Utilization • Coverage • Equity • Efficiency
- 23. 23 1.2 Processes (cont.) • Efficacy – Does the agent or intervention work under ideal laboratory conditions? – Tested where all conditions are controlled • Effectiveness – How well does the agent or intervention work in a real- life situation? • Efficiency – Is it possible to achieve goals in a cheaper or better way? – Cost-benefit • Not only cheaper, but less discomfort, pain, disability, social stigma – If not effective, not worth looking for efficiency Randomised Controlled Trials
- 24. 24 1.3 Outputs • Were services provided appropriate, relevant, & adequate? – Service quantity – Service quality – Efficiency of services – Acceptability to community – Socio-economic effects – Distributional effects – Gender effects
- 25. 25 1.4 Outcomes • User satisfaction • Complete health • Residual disease • Residual disability • Death
- 26. 26 1.4 Outcomes(cont.) • Measures of outcome: –Quantifiable –Easy to define and diagnose –Lend itself to standardization
- 27. Indicators & Criteria for evaluation • Indicators – Variables that measure change – Direct or indirect • Validity • Measures what it is supposed to measure • Reliability • Results the same whoever measure it • Sensitivity • Sensitive to changes in situation or phenomena 27
- 29. Indicators & Criteria for evaluation • Health Policy indicators • Social & Economic indicators • Indicators of provision of health care • Indicators of coverage • Health status indicators 29
- 30. Indicators & Criteria for evaluation • Standards by which actions are measured –Technical e.g. drinking water standards • Leads to judgment being made • Need available information 30
- 31. Pertinent questions • Where indicators & criteria not available • Ask about the activity to be evaluated 31
- 32. Information support • Evaluation is based on – valid, – relevant – & sensitive information • Limit amount –Historical, social, political, economic, scientific, technological, demographic, epidemiological, organizational, legislative etc 32
- 33. 33 When to evaluate • Summative evaluation –At completion of project or activity –Did it achieve its objectives? • Formative evaluation –Activity still being carried out –Can it be modified? –Monitoring
- 34. 34 Frequency of Evaluation • Continuing process –Monitoring • Results need to be summarized & reported at specified intervals
- 35. 35 2. WHO framework for evaluation • Specify the particular subject for evaluation • Ensure information support • Verify relevance • Assess adequacy • Review progress • Assess efficiency • Assess effectiveness • Assess impact • Draw conclusions, • Formulate proposals
- 36. 36 2.1 Specify the subject for evaluation • What is to be evaluated? – Project, Programme, service, institution • Organizational level? • Purpose of evaluation? Why? • Constraints? • Options existing for decision? • Reported to whom?
- 37. 37 2.2 Ensure information support • What are information requirements? –Subject of evaluation, relevance, –Adequacy of problem definition, –Adequacy of programme or project formulation, Progress, Efficiency, Effectiveness, Impact • What are available sources of information? • Is information adequate? –Need special studies or surveys?
- 38. 38 2.3 Verify relevance • Rationale of health policies, programmes, activities, services & institutions –Needs based –Justified in social terms –Related to objectives
- 39. 39 2.4 Assessment of adequacy • Problem clearly defined –Public health importance • Properly formulated –Provision for planning, management & evaluation –Objectives clearly stated –Plan of action –Alternative approaches considered –Indicators & criteria
- 40. 40 2.5 Review progress • Personnel trained • Coverage of service • Time-frames • Equipment & supplies • Schedules kept • Funds spent with those budgeted
- 41. 41 2.6 Assess efficiency • Results obtained in relation to efforts made –Technical efficiency –Allocative efficiency •Are funds directed to activities that will produce the greatest gain? –Cost efficiency • Quality of Care –Safe, contributes to good health, meets user expectations, best use of finite health care resources
- 42. 42 2.7 Assessment of Effectiveness • Analysis of attainment of objectives & target met • Health problem reduction • Doing the right thing
- 43. 43 2.8 Assess impact • Improved overall health & socio- economic situation & quality of life –vs just meeting objectives
- 44. 44 2.9 Draw conclusions& formulate proposals for future action • Summarize he objectives, approaches, methods & results of the programme • Summarize supporting information • Draw conclusions • Formulate proposals for future action
- 45. 45 3. Framework for Program Evaluation in Public Health • CDC - MMWR 1999;48(No.RR-11):1-35
- 46. 46 CDC Evaluation Steps Standards Utility Feasibility Propriety Accuracy Engage stakeholders Describe the Programme Focus the evaluation design Gather credible evidence Justify conclusions Ensure use & Share lessons learned
- 47. 47
- 48. CDC Steps in Evaluation Practice 3.1 Engage stakeholders –Those involved in programme operations –Those served or affected by the programme –Primary users of the evaluation 48
- 49. CDC Steps (cont.) 3.2 Describe the programme –Need • Problem or opportunity being addressed –Expected effects –Activities • What it does to effect change 49
- 50. CDC Steps (cont.) – Resources • Time, talent, technology, equipment, information, money & other assets – Stage • Planning, implementation & effects – Context – Logic model • A picture 50
- 51. 51
- 52. CDC Steps (cont.) 3.3 Focus the evaluation design – Purpose – Users (of the evaluation findings) – Uses – Questions – Methods – Agreements 52
- 53. CDC Steps (cont.) 3.4 Gather Credible evidence – Indicators: address criteria that will be used to judge the programme – Sources: persons, documents or observations that provide information for the evaluation – Quality: appropriateness & integrity of information – Quantity: amount of information gathered – Logistics: methods, timing and physical infrastructure for gathering and handling evidence 53
- 54. 54 CDC Steps (cont.) 3.5 Justify conclusions – Standards – Analysis/synthesis – Interpretation – Judgment – Recommendations 3.6 Ensure use & share lessons learned – Design – Preparation – Feedback – Follow-up – Dissemination
- 55. 55 3.7 Standards for Effective Evaluation • Utility –Serve the information needs of the users • Feasibility –Be realistic, prudent, diplomatic & frugal • Propriety –Behave legally, ethically • Accuracy –Reveal & convey technically accurate information