Reflections on monitoring a large-scale civil society WASH initiative: Lessons for sector monitoring and potential contributions from NGOs
Download as PPT, PDF0 likes394 views
The document discusses lessons learned from donor monitoring for sector monitoring within a large-scale civil society WASH initiative. Key lessons highlight the importance of a learning focus, balancing specificity and flexibility in indicators, and utilizing an actor-centered theory of change. It also outlines various NGO strategies for supporting sector monitoring that have yet to be widely adopted.
Reflections on monitoring a large-scale civil society WASH initiative: Lessons for sector monitoring and potential contributions from NGOs
- 1. THINK CHANGE DO REFLECTIONS ON MONITORING A LARGE- SCALE CIVIL SOCIETY WASH INITIATIVE Lessons for sector monitoring and potential contributions from NGOs Tim Brennan and John McKibbin A/Prof Juliet Willetts, Bruce Bailey, Dr Paul Crawford IRC Symposium 2013 Addis Ababa Griffin nrm
- 2. Key messages • Three lessons for sector monitoring from donor monitoring: 1. Investing in a ‘learning focus’ as part of sector or donor monitoring is critical 2. Tensions between specificity and flexibility in choosing indicators and frameworks must be addressed carefully 3. An actor-centred ‘theory of change’ provided a helpful, readily understandable structure • A wide range of strategies are available to NGOs to support sector monitoring (but so far are little adopted)
- 3. A large-scale civil society WASH fund… $32.5 million 11 CSOs 21 countries 782,000 people with toilets 45 activities $200K- >$1.6m 564,000 people with water
- 4. Our role in the Fund… diverse activities of the monitoring review panel (MRP) … • Engaged for life of the program • Designed the performance framework and reporting requirements • Conducted analysis and synthesis – learning papers • Undertook monitoring of Fund activities • Provided an M&E capacity building role • Undertook two independent evaluations of the Fund
- 5. Lesson 1 from donor monitoring: Invest in a learning focus • ‘Developmental evaluation’ Developmental approach – what did we do?: evaluation adapts evaluation (or • Collected ‘real-time data’ to inform on- monitoring) to going decision making and adaptations “emergent and • Included descriptive free-form case dynamic realities studies as well as indicators in complex environments” • Used evaluation questions (not just (Patton, 2011, indicators) p1). • Facilitated, structured learning ‘events’ • On-going one-on-one feedback to NGOs • Learning ‘products’ that synthesise lessons across contexts
- 6. Lesson 2 from donor monitoring: Balance specificity and flexibility • Why is specificity needed in sector monitoring? • Why is flexibility needed in sector monitoring? • How can these needs be balanced?
- 7. Flexible guidance on what to monitor or report 19 ‘result areas’ 7 mandatory indicators We achieved balance by defining some indicators very tightly, and making space in other ways for more diverse, flexible inputs
- 8. Lesson 3 from donor monitoring: Actor- centred theory of change was helpful • Monitoring is usually about measuring ‘change’ • ‘Theories of change’ simplify reality to make it easier to measure, communicate and understand how we think change happens • We can then also test our assumptions about how change happens
- 9. Theory of change behind the performance Poor communities arrangements & households Institutional & community partners CORE OUTCOMES: Outcome 1: increased access to ENABLING sanitation CSO OUTCOMES: Outcome 2: Delivery Team Outcome 4: improved increased access to WASH governance water Outcome 5: Outcome 3: DELIVERABLES: strengthened local improved hygiene capacity behaviour E.g. training, mentoring, advocacy, Outcome 6: improved technical advice, gender equality campaigns etc. Outcome 7: improved WASH evidence base
- 10. NGO roles to support sector monitoring: a ‘strategy map’ Strategy Causal Persuasive Supportive I-1 I-2 I-3 Focused on a particular individual or group E-1 E-2 E-3 Focused on the enabling environ- ment Strategy map was developed by IDRC as a
- 11. Strategy Causal Persuasive Supportive I-1 I-2 I-3 Focused Direct role in Providing awareness Providing frequent, on a particular monitoring own raising, education or sustained, on-going individual direct specific training to mentoring and support; or group implementation community members or multipurpose activities. or other partners. capacity building; or developing support structures, committees & networks. E-1 E-2 E-3 Focused Engaging in Dissemination of Building partnerships, on the enabling policy dialogue, information to broad providing collective environ- directly causing audience; support and promoting ment changes in persuasive networking and incentives, rules; environment for a coordination; also playing an specific behaviour or supporting higher levels advocacy role. attitude; conducting of government or local workshops/conf’s. research networks.
- 12. Examples of NGO roles and their benefit to sector monitoring • Direct monitoring role (I-1 strategy map) “one NGO in Bangladesh showed that government (Department of Public Health Engineering) data for sanitation coverage of 38-58% in three ‘upazilas’ in 2009 was actually at 5-28% when the NGO conducted their baseline study in 2010, with these large differences gaining attention from relevant government staff”
- 13. Examples of NGO roles and their benefit to sector monitoring • Systematic strategies to build local government expertise in monitoring (I-3) “one NGO working in Indonesia and Bangladesh directly supported existing government monitoring systems for sanitation by working in a direct support role for local government”
- 14. Examples of NGO roles and their benefit to sector monitoring • Document and share own learning/innovations and promote their uptake(E-2) “An NGO working in Bhutan achieved adoption of elements of their project monitoring framework which used a qualitative information scale for examining the hygienic status of toilets, and also includes a scale for examining handwashing with soap. Their ideas were taken up and integrated into government monitoring systems, including Ministry of Health’s Annual Survey in February/March each year, and inclusion of indicators to sanitary access and hygienic toilet usage in the next Five Year Development Plan from mid-2013 as Key sector Result Areas” (Halcrow, pers comm. 2013)”
- 15. So what? What needs to change for NGOs to better support sector monitoring? Government staff should: • Consider and communicate the range of roles which NGOs might play in support of national initiatives Program designers (both NGOs and donors) should: • Make efforts to adopt strategies explored in this presentation, particularly considering strategies not yet in use M&E specialists: • Should align monitoring and evaluation systems of WASH projects to sector monitoring, instigate the sharing of information with government
- 16. Key messages • Three lessons for sector monitoring from donor monitoring: 1. Investing in a ‘learning focus’ as part of sector or donor monitoring is critical 2. Tensions between specificity and flexibility in choosing indicators and frameworks must be addressed carefully 3. An actor-centred ‘theory of change’ provided a helpful, readily understandable structure • A wide range of strategies are available to NGOs to support sector monitoring (but so far are little adopted)
- 17. Thank you Contacts: Juliet.Willetts@uts.edu.au, pcrawford@aid-it.com, bbailey@griffin-nrm.com.au Griffin nrm
Editor's Notes
- #12: Animated version for presentation