Evaluations to improve organisational effectiveness
- 1. Theme: Evaluation Use, Communication and Reach. Use of Evaluation results to enhance organizational effectiveness: Do evaluation findings improve organisational effectiveness? Innocent Karugota Muhumuza Planning, Monitoring and Evaluation Officer. Caritas Switzerland-Uganda B.A, Dip Disaster Mgt, PGD M&E, Masters M&E (In-progress) September 28, 2016 Maastricht, Netherlands
- 2. Content Defining Monitoring and Evaluation (M&E) Why Conduct Evaluations and What is Use Typical Use of Evaluation (Case Study) Factors influencing Use Conclusions
- 3. Defining Monitoring and Evaluation EvaluationMonitoring A continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds- OECD, 2002 A time-bound exercise that attempts to assess systematically and objectively the relevance, performance and success of ongoing and completed programs and projects- UNDP, 2002
- 4. Why Conduct Evaluations? According to Development Assistance Committee (DAC), Evaluations are conducted to assess: • Effectiveness How well objectives are met • Efficiency How well resources have been used • Relevance How appropriate the intervention is/was to meet the needs of the beneficiaries • Impact The intended and uninteded benefits of the intervention • Sustainability The ability/potential of interventions’ impact to continue long after the end of the funding or involvement of the donor
- 5. What is Evaluation Use/ Utilisation? Numerous definitions of Evaluation Use exist Evaluation Use and Evaluation Utilisation used interchangeably to mean: • The application of evaluation processes, products or findings to produce an effect (Johnson, Greenseid et al, 2009). • How real people in the real world apply evaluation findings and experience and learn from the evaluation process (Patton, 2013).
- 6. Types of use Many types of use exist, including; • Instrumental- when decision makers use the evaluation findings to modify the object of the evaluation (Shulha & Cousins, 1997) • Conceptual- An evaluation helps program staffs understand the program in a new way-(Weiss, 1979) • Symbolic- When the organisation or individuals use the mere existence of evaluations and not any aspects of the results to persuade or to convince
- 7. Process use incorporates features from instrumental use, enlightment and conceptual use Types of use, Cont’d • Legitimisation-the evaluation is used to justify current views, interests, policies or actions • Enlightment- evaluation findings add knowledge to the field and so may be used by any one (Weiss, 1979) • Process-....cognitive, behavioural, program and organisational changes resulting from engagement in the evaluation process and thinking evaluatively (Patton, 2003). Influences thinking and behaviour and program/organisations’ procedures and culture
- 8. Evaluation use in practice Acase of Advance Afrika (AA)-Uganda The project; Youth Entrepreneurship Enhancement Project (YEEP) • Was a pilot for one year (March 2004-Feb 2015) • Focus: Reformation, Rehabilitation and Re-integration • Covered prisons in only 3 district of Northern Uganda • Implemented by AA staff, Staff of Uganda Prisons Service, Students of Gulu University-Uganda
- 9. The Evaluation: • Involved; AA staff, Staff of Uganda Prisons Service (UPS), Students of Gulu University and Ex-convicts (beneficiaries) • Facilitated by an external Evaluator • 8 recommendations were made: • Conduct refresher training for all UPS trainers • Strengthen Advocacy • improve training manual • Develop a standard reporting guide for the project • UPS should be involved more in following up Ex-Convicts • Re-define the project target beneficiaries (geographical boundaries) • involve more (number of) UPS staff for wider support to the project • Build capacity of AA staff and co-implementers
- 10. Evaluation use in practice, Cont’d From the recommendations: • Two full-fledged projects were developed 1. Advancing Youth Entrepreneurship (AYE)-(2015-16) 2. Social Re-integration and Economic Empowerment (SREE)-(2016-18) • Project covers 26 prisons in Acholi and Lango sub-regions of Nothern Uganda
- 11. Factors influencing Use Quality External Factors Organisational Factors Utilisation Staff & Organisation • Culture • Structure • Systems • Policies • Budgets • Purpose • Design • Planning and timing • Dissemination and • Credibility of evidence • Professional Bodies/Associations • Project Beneficiaries • Donors • Protection of funding and reputation Environment Technology Legal
- 12. Conclusions Quality factors. It is key that, the design of the evaluation as much as possible involves key stakeholders not as mere repondents but as active participants in the process. This helps to enhance credibility of the evidence and ownership of the results. Organisational Factors. Even when it is good to have structures and policies in the organisation, care must be paid on how this affects learning and relations. External Factors Pressure from the external influencers e.g donors must be kept in check and organisations be granted opportunity to learn and improve from failure and also consolidate the gains made. This means that the external actors ought to support organisations to improve rather than instilling fear of the ‘‘unknown’’
- 13. Asante Sana!