Cida lessons from evaluations

Canadian International Development Agency (CIDA)
200 Promenade du Portage Gatineau, Québec K1A 0G4
Canada
Telephone:
819-997-5006 / 1-800-230-6349 (toll-free)
For the hearing- and speech-impaired:
819-953-5023 / 1-800-331-5018 (toll-free)
Fax: 819-953-6088
Website: www.cida.gc.ca/evaluations-e
E-mail: info@acdi-cida.gc.ca
© Her Majesty the Queen in Right of Canada, 2012
Cette publication est aussi disponible en français : L’ACDI apprend : Leçons tirées des évaluations 2011-2012
Printed in Canada

1
Contents
Preface ......................................................................................................................................... 2
Lessons Learned in 2011-2012 .................................................................................................. 3
1. Map the route, and adjust for road conditions .......................................................... 4
2. Journey together .......................................................................................................... 6
3. Look for the exit ........................................................................................................... 7
4. Focus on the journey, not just the destination ......................................................... 8
Looking forward: the importance of resilience......................................................................10
Conclusion ................................................................................................................................11
Annex: Highlights of Evaluations ............................................................................................12
Evaluations led by the Evaluation Division
1. A Review of the Evidence of the Effectiveness of CIDA's Grants and Contributions 2005/06- 2010/11
2. Country Program Evaluation - Ukraine
3. Evaluation of CIDA’s Regional Inter-American Program (2004-2005 to 2009-2010)
4. Evaluation of CIDA's Peru Program 2005-2010
5. Honduras Country program Evaluation 2002-2010
6. Review of the World Food Programme's Humanitarian and Development Effectiveness 2006- 2011
7. Corporate Evaluation of CIDA's Humanitarian Assistance
8. Review of the United Nations Development Programme's Development Effectiveness
Evaluations led by Program Branches
9. Nepal Peace Building from Below Project
10. Sri Lanka Local Initiatives for Tomorrow 2
11. Ukrainian Civil Service Reform Project
12. Microfinance Component of the Agricultural Development in Mine-affected Areas of Cambo- dia (ADMAC) Project
13. Turquoise Mountain Regenerating the Murad Khane District of Kabul (RMKD)
14. Projet d’appui à la surveillance épidémiologique intégrée - prolongation (PASÉI 2 – prolongation)
15. Évaluation conjointe de «l’intervention de Viva Rio dans la zone de bel air» en Haïti – ap- puyée par l’ACDI, le MAECI/GTSR et NCA
16. Observatoire du Sahara et du Sahel (OSS)
17. Evaluation of the Micronutrient Initiative
18. Strengthening Access to Justice through Legal Sector Development
19. International Legislative Audit Assistance Program

2
Preface
About evaluation
Evaluations are a tool for managers to review achievements, learn from experience, be accountable, and make better decisions on the basis of evidence.
Evaluation at CIDA
All Government of Canada departments, including CIDA, are required to evaluate all direct program spending over a five-year period. For CIDA, this means evaluating some $15 billion in programming. CIDA programs are evaluated against both Cana- dian1 criteria such as relevance, effectiveness, efficiency, and economy; and international2 criteria, which add sustainability and impact. Operationally, CIDA’s Evaluation Division evaluates CIDA programs and provides support to branch-led evaluations of specific projects. CIDA’s evaluation function is one of the strongest in the Government of Can- ada.
About this report
Lessons drawn from evaluations can improve understanding of what constitutes good development, and what works and what does not, for both CIDA staff and partners who design and implement international development projects. Incorporating these lessons into our operations will also give Canadian taxpayers better value for their tax dollars.
This first report presents a set of key lessons dis- tilled from a sample of evaluation reports completed in 2011–2012 organized according to themes that emerged from our review. Setting a one-year review time frame ensures our information is new and allows us to provide timely feedback to CIDA.
For this report, we reviewed 19 evaluations (each summarized in the annex) covering a broad range
of CIDA programs and projects: 12 covering Geo- graphic Programs Branch, 4 covering Multilateral and Global Programs Branch, 2 covering Partner- ships with Canadians Branch, and the Agency-wide Review of Evidence of the Effectiveness of CIDA’s Grants and Contributions. We analyzed the evaluations to identify lessons, and further analyzed the lessons to identify the most relevant factors for CIDA. Focus group discussions helped pinpoint relevant issues.
This year’s lessons
The metaphor of a road trip encapsulates lessons identified from this year’s evaluation reports.
1. Plan before leaving on a trip, and make adjustments during the trip for road conditions and traffic. 2. Travel with others to enrich your trip and avoid pitfalls along the way. 3. As you drive, look for the proper exit, and move into position to take it. 4. Fo- cus on the journey, not just the destination.
CIDA’s next challenge: resilience
One cross-Agency issue that emerged from this year’s evaluations is the issue of resilience. Build- ing resilience protects the gains from CIDA’s investments over the years, and is also more cost- effective than rebuilding.
In conclusion
In the business of development, CIDA and its partners work together toward shared goals—a journey together. By inviting us to learn from our successes and to seek out opportunities for improvement, this report aims to make that journey as well marked, swift, and successful as possible.
Caroline Leclerc,
CIDA’s Head of Evaluation
1 Directive on the Evaluation Function. Canada 2009. (http://www.tbs-sct.gc.ca/pol/doc- eng.aspx?id=15681&section=text)
2 DAC Criteria for Evaluating Development Assistance. OECD 1991.
(http://www.oecd.org/dac/evaluationofdevelopmentprogrammes/daccriteriaforevaluatingdevelopmentassistance.htm)

3
Lessons Learned in 2011-2012 CIDA “Picture of the Year,” for 2007 which captures feeling a sweltering day in 2006 on beach Mombasa, Kenya, as fishermen push boat across sand to launch it into Indian Ocean. Copyright CIDA, Wendell Phillips.

4
1. Map the route and adjust for road conditions
Use established tools to seize opportunities and address risks
before, during, and after implementation Plan investments with specific outcomes. Make adjustments along the way to improve performance. Discuss or coordinate efforts for better results.
Although the evaluations reviewed are generally positive in their assessment of performance and the results achieved, they also point to some ongoing challenges with design, performance management, and coordination of projects and programs. This finding applies even though most projects undergo a lengthy review prior to approval. The review aims to ensure that investments are aligned with local needs and Canadian priorities, design is adequate, results are sustainable and measurable, and progress and expenditures are appropriately reported.
Plan investments with specific outcomes
When specific and measurable outcomes are not clearly defined, performance is harder to measure, risks can be overlooked, and projects are not as effective as they could be.
Justice sector (SAJEA)3 and Nepal peacebuilding4 evaluations reiterate the importance of setting clearly defined performance-based targets. Another evaluation (ILAAP)5 notes that desired outcomes should be clearly defined in consultation with partners, and recommends holding a single point of contact responsible for the coordination and achievement of results.
In Honduras, CIDA adopted a flexible strategy with multiple aid-delivery channels (project- and program-based approaches) to mitigate risks. That strategy helped avoid a complete interruption in
delivery during the 2009 political crisis. In contrast, donors, including CIDA, enthusiastically embarked on the Education for All pooled fund before present- day risk-management practices and guidance on program-based approaches. This experience taught donors to ensure that adequate governance and institutional arrangements were in place before embarking on program-based approaches.6
An evaluation in Sri Lanka (LIFT2)7 goes a step further, and notes that strategic planning and awareness building about opportunities during the design phase can result in a better connection between activities. LIFT2 also notes that identifying indicators (notably gender-specific indicators) at the planning stage is much better than revisiting indicators during the project. The importance of strategic planning was echoed by the World Food Pro- gramme (WFP) review8, which also noted the importance of proper targeting, a customized design based on strategic choices, and the establishment of clear objectives and prioritization.
Adjust along the way to improve performance
Several evaluations recommended making adjustments along the way to address management challenges and to adapt to an evolving context. Manag- ing projects to ensure performance can be difficult, and not extending projects to consolidate gains can reduce project effectiveness and sustainability.
3 Summative Evaluation: Canadian Bar Association 2009–2011 Program Phase 1: Strengthening Access to Justice through Legal Sector Development, October, 2011
4 Final Evaluation Report: Nepal Peacebuilding from Below (PBB) Project, November 2011
5 Evaluation Report: Evaluation of the International Legislative Audit Assistance Program, May 2011
6 Honduras Country Program Evaluation 2002–2010, 2011
7 Report on Findings: Mid-term Evaluation
of the Local Initiatives for Tomorrow–2 (LIFT2), February 2012
8 Humanitarian and Development Effectiveness Review of the World Food Programme 2006–2011, 2012

5
CIDA’s long-running support to PASEI9 achieved results, but gains were limited by poor performance management. Its evaluation recommended that CIDA ensure that projects utilize results-based- management tools such as logic models, performance measurement frameworks, and risk analyses; otherwise, projects may fail to learn lessons drawn from experience. The evaluation also recommended developing a sufficiently long time horizon to achieve and sustain gains: it noted that important results were achieved after the project was extended, allowing significant prior gains to be con- solidated. Similarly, LIFT2 noted that the time frame for project implementation was not long enough to achieve the transformative change envisioned.
The grants and contributions (Gs&Cs) review10 puts it more bluntly, noting that relevance is not static, and that country programs and activities should be reviewed in light of changes to the local context and the strategies of the recipient country and other donors. This finding was echoed in the Micronutrient Initiative (MI) evaluation,11 which recommended revisiting and clarifying performance-measurement indicators, both to ensure that indicators are measurable as
well as clarifying performance expectations.
Discuss or coordinate efforts for better results
Dialogue and coordination can pave the way to effective design and successful readjustments. Evaluations note that success in a complex environment often requires participation from multiple participants, including governments, civil society and donors, all of which is supported by dialogue and coordination.
The Gs&Cs review notes that coordination is particularly beneficial in fragile states, but requires flexibility, harmonization, risk analysis, risk toler- ance, and experienced staff. Some other examples from this year’s evaluations where dialogue and coordination are useful include: identifying what other organizations are contributing in the intervention areas in order to better design country-level plans (MI); addressing intra-community conflicts successfully in Haiti (Viva Rio); 12 combining resources with partners for better interventions (LIFT2); and adopting a holistic approach to building capacity (ILAAP).
9 Évaluation de fin de projet : Projet d’appui à la surveillance épidémiologique intégrée - prolongation (PASÉI 2 – prolongation), septembre 2011
10 A Review of Evidence of the Effectiveness of CIDA's Grants and Contributions 2005/06–2010/11, 2011
11 Evaluation of the Micronutrient Initiative, March 2012
12 Évaluation conjointe de «l’Intervention de Viva Rio dans la zone de Bel Air» en Haïti – appuyée par l’ACDI, le MAE- CI/GTSR et NCA, mai 2011 © CIDA/PEDRAM PIRNIA

6
2. Journey together
Build and strengthen relationships to improve relevance, performance, and sustainability
Evaluations indicate that strong relationships are associated with improved relevance, performance, and sustainability of results. Building and strengthening relationships can help improve effectiveness by dealing with contextual challenges. For example, in a fragile or conflict-affected area, relationships built between various stakeholders can reduce vul- nerabilities related to the political situation.
The Gs&Cs review notes several examples of effectiveness from building and strengthening relationships. In the case of Bangladesh, CIDA’s leadership on gender equality and partnership with civil society and government achieved substantive results. In Ethiopia, dialogue and coordination with the government led to local ownership of poverty- reduction priorities and the government investment necessary to sustain developmental progress.
The Honduras country program evaluation noted that in a country where governance and public administration capacity are weak, working with local stakeholders allowed CIDA to gain in-depth knowledge of local issues and develop a relationship of trust that favoured the Agency as a whole. The WFP review echoed this finding, noting that it was able to achieve objectives by working in partnership and in a participatory manner with host governments, other UN agencies, and local communities.
Other examples where strong relationships mat- tered are: successful implementation in Kabul, Afghani- stan, by building and maintaining a constructive relationship (RMKD); 13 better results when strong technical approaches promoted a common vision (OSS); 14 and project support from building good relationships with divisional and local government officials in Sri Lanka and Ukraine (LIFT2 and UCS- HRM).15
LIFT2 notes that naming appropriate contacts and service providers to address specific issues along with training and information about how to best ar- ticulate needs should serve as a model for other projects.
13 Final Report: Summative End of Project Evaluation: Tur- quoise Mountain Regenerating the Murad Khane District of Kabul (RMKD), March 2012
14 Évaluation sommative de fin de projet : Projet d’appui à l’Observatoire du Sahara Sahel (OSS), avril 2011
15 Evaluation Report: Summative Evaluation of the Ukraine Civil Service Human Resources Management Reform Project (UCS-HRM), September 2011 © CIDA/PETER BENNETT

7
3. Look for the exit
Plan to phase out, ensure staff capacity, and enable horizontal learning Plan and implement exit strategies. Build capacity. Promote peer-to-peer learning.
In addition to achieving the expected outcomes in the first place, initiatives should aim to sustain outcomes. This focus on sustainability requires up- front planning, supporting partners to develop necessary capacity, and enabling horizontal learning and support mechanisms.
Plan and implement exit strategies
Several evaluations, including those for the WFP and United Nations Development Programme (UNDP),16 note that strong phase-out plans and the integration of sustainability into project design facili- tate the sustainability of results. The RMKD evaluation noted that the presence of an exit strategy from the beginning of implementation informed the me- dium-term thinking of project management and effective execution of plans. Still, the evaluation noted that having an exit strategy was not enough: the project had to deal with unforeseen contingencies and essentially had to extend elements of its intervention to ensure sustainability.
The Honduras evaluation noted that in the period examined by the evaluation, contextual challenges outside of CIDA’s control reduced overall sustainability. The most sustainable projects were those that aimed to build capacity and human capital, with explicit phase-out plans.
Build capacity
Exit strategies and program design must aim to support partner capacity to ensure results are sustainable. The WFP review noted that capacity- building efforts, particularly training, were a means to develop institutional and community capacity for sustainability. The review also noted that WFP itself
faced weaknesses in managing its capacity, spe- cifically technical expertise and staff turnover. The Nepal peacebuilding evaluation touched upon the importance of developing staff capacity and mini- mizing staff turnover in improving results. It suggested tracking partner capacity from the outset and adopting a differentiated strategy based on partner capacity. The SAJEA evaluation noted that the coordination of national working groups required both will and capacity.
Promote peer-to-peer learning
Supporting learning between peers can help provide capacity and support after the project ends. The Nepal peacebuilding evaluation noted that the project’s “peace networks” gave various community organizations the chance to interact and learn from each other. Weaker and more remote partners benefited from exposure to more capable partners. Other examples that echo these lessons are: creating support networks early during project implementation to improve sustainability (AD- MAC); 17 linking donor country and provincial audit offices to provide mentoring and ongoing support (ILAAP); exchanges, learning, and openness to doing things differently while respecting cultural dif- ferences and building on similarities (Viva Rio); and drawing on technical expertise and experience at the regional and country levels to dissemi- nate lessons learned and best practices (MI).
16 Development Effectiveness Review of the United Nations Development Programme 2005–2011, 2012
17 Final Evaluation Report: Microfinance Component of the Agricultural Development in Mine-affected Areas of Cambodia (ADMAC) Project, July 2011 © ACDI-CIDA/PETER BENNET

8
4. Focus on the journey, not just the destination
Decentralize decisions and delegate for better results, and create mechanisms for organizational improvement Be flexible to improve efficiency and effectiveness Be flexible to improve relevance Enable organizational improvement
Evaluations note that decentralizing decision making to the field allows programming to be more responsive, efficient, and effective. Decentralization can also enable organizational empowerment, learning, and evolution both within and outside CIDA. Finally, decentralization can empower both CIDA staff and partners to address the challenges raised earlier in this report: making better adjustments to the context, building and strengthening relationships, and planning and implementing exit strategies.
One objective of decentralization is flexibility. The flexibility of staff concerning elements such as program activities, timing, or even partner organizations, allows for responsive and effective management, as noted in the WFP review. Flexibility can lower costs by allowing innovative solutions to emerge and by reducing administrative overhead. Flexibility can also improve relevance.
Another objective of decentralization can be to enable organizational empowerment, learning, and evolution. Empowered stakeholders can contribute to organizational improvement and hold their management (or government bodies) accountable. This empowerment provides an opportunity for improvement among both CIDA and its partners, and is not only in line with commitments from the Paris Declaration and other summits, but draws together several strands of CIDA’s work.
The WFP review noted that the use of local resources was an innovative solution to logistical challenges and resulted in greater efficiency overall. By identifying opportunities for greater local procurement, the WFP supported local economic growth while addressing its humanitarian mandate. Fundamentally, this approach would not work with- out delegating decision-making power to field staff, while following organizational parameters and policies to ensure accountability.
Be flexible to improve efficiency and effectiveness
The ILAAP evaluation noted that on-site and online training could reduce costs and increase reach while reducing travel time for participants. The Gs&Cs review noted that transaction costs can be reduced only if CIDA plans and manages its participation in joint donor activities strategically.
The Peru evaluation18 noted that small, responsive, flexible short-term projects are useful complements to long-term projects because they allow innovation and experimentation, as well as targeted and timely interventions. Small projects also provide low-cost, low-risk opportunities for developing learning and innovation clusters and for producing synergy and results that are greater than the sum of the parts.
Be flexible to improve relevance
Several evaluations highlighted the importance of making decisions flexibly and closer to beneficiaries. These include: improving on design during implementation (Nepal peacebuilding); balancing between constancy and the demands of a changing environment (Ukraine19); and adding value by using a tailored subregional approach to fit the context and better engage stakeholders (Inter-American20 and MI); and
18 Evaluation of CIDA's Peru Program 2005-2010, 2012
19 Ukraine Country Program Evaluation 2004 – 2009, 2011

9
accounting more effectively for context through flexible program design (LIFT2).
Enable organizational improvement
Evaluations noted the importance of incorporating feedback effectively to guide the organization. This feedback can come from staff, partners, or other stakeholders.
The MI evaluation suggested that a review and clarification of roles at various levels of centraliza- tion would be welcome, and might provide opportunities for field staff to provide input into the organization and to be heard.
The OSS evaluation noted the importance of a responsive governance structure in a regional body to implement actions and respond to member needs. The SAJEA evaluation highlighted the importance of decision-making authority for the working groups created by the project to be successful.
The evaluation of CIDA’s humanitarian assistance21 expands this lesson to include CIDA by highlighting the importance of mechanisms to systematically integrate information and share lessons learned to improve effectiveness at headquarters and in the field.
20 Evaluation of CIDA’s Regional Inter-American Program (2004–2005 to 2009–2010), 2011
21 Corporate Evaluation of CIDA's Humanitarian Assistance 2005–2011, 2012 © CIDA/PIERRE ST-JACQUES

10
Looking forward: the importance of resilience
One cross-Agency issue that emerged from this year’s evaluations is the issue of resilience. The evaluation of CIDA’s humanitarian assistance recommended that CIDA develop a systematic, integrated approach to supporting prevention and risk reduction, as well as recovery and transition to development. The 2012 OECD DAC Peer Review of Canada22 echoed this finding, suggesting improvements to Canadian efforts in building resilience and supporting post-crisis recovery.
Resilience refers to the ability of individuals, households, governments, regions, and systems to mitigate, resist, absorb, and recover from the effects of shocks and disasters in a timely, sustainable, and efficient manner.
Building resilience before a disaster strikes has the potential to save more lives, better use resources, and guard against future crises. Prevention is often less costly than disaster relief and response, and building resilience can provide a lasting response to the cycle of shocks and disasters that regularly af- fect particularly vulnerable countries and regions.
At any level of intervention, effective preparation can mitigate the impact of these events. Risk management and systematic planning for disaster risk reduction are key elements of preparation. CIDA could better integrate resilience into its investments by systematically identifying evolving risks and by responding quickly in response to early warning signals.
The humanitarian-assistance evaluation describes a flexible mechanism that can respond as needed. The Government of Ethiopia’s Productive Safety Net Program (PSNP) is a development program that includes a risk-financing mechanism to provide emergency support to families. Although CIDA supports the PSNP overall, it is not able to support the risk financing mechanism, which integrates humanitarian and development programming, because it falls outside the mandates of both CIDA’s
humanitarian assistance and development programs. The World Bank and the United Kingdom’s Department for International Development fund the risk-financing mechanism.
Each of the lesson areas identified in this report also offers an opportunity for CIDA to build resilience. CIDA should seize opportunities and identify risks, build relationships and work collaboratively, prepare phase-out and contingency strategies, support partner capacity building and learning, and decentralize decision-making authority to allow for more flexible, efficient, and self-governed responses.
Fundamentally, sustainable development builds the foundation for resilience: improving levels of social and economic development reduces vulnerability to shocks and disasters.
CIDA notes in its 2011/2012 Departmental Per- formance Report that it is “continuing to work on developing a comprehensive and holistic approach to disaster risk reduction and recovery situations.” The Departmental Performance Report also noted examples from other CIDA evaluations and studies of the “need to focus on long-term strategies and building resilience.”
22 OECD’s Review of the Development Co-operation Policies and Programmes of Canada, 2012

11
Conclusion
Evaluations give us an opportunity to improve our policies and programs on the basis of what we have learned.
This report is the very first of its kind at CIDA. It at- tempts to identify and summarize lessons from evaluations completed over the past fiscal year. The notable successes and challenges revealed by these evaluations relate to the common experience of working together toward a common goal in a complex environment. Such work requires good planning, management of contingent circumstances, and collaboration among multiple individuals and organizations.
One-page summaries of all the evaluations reviewed are appended. They add context and detail to this narrative.
Adopting the lessons in this report is both an opportunity and a challenge. Future evaluations will have the opportunity to assess the extent to which they have been effectively learned

12
Annex: Highlights of Evaluations
Evaluations led by the Evaluation Division
Agency-wide 1. A Review of the Evidence of the Effectiveness of CIDA's Grants and Contributions 2005/06–2010/11
GPB 2. Country Program Evaluation – Ukraine
3. Evaluation of CIDA’s Regional Inter-American Program (2004–2005 to 2009–2010)
4. Evaluation of CIDA's Peru Program 2005–2010
5. Honduras Country program Evaluation 2002–2010
6. Review of the World Food Programme's Humanitarian and Development Effectiveness 2006-2011
7. Corporate Evaluation of CIDA's Humanitarian Assistance
8. Review of the United Nations Development Programme's Development Effectiveness
Evaluations led by Program Branches
GPB 9. Nepal Peace Building from Below Project
10. Sri Lanka Local Initiatives for Tomorrow 2
11. Ukrainian Civil Service Reform Project
12. Microfinance Component of the Agricultural Development in Mine-affected Areas of Cambodia (ADMAC) Project
13. Turquoise Mountain Regenerating the Murad Khane District of Kabul (RMKD)
14. Projet d’appui à la surveillance épidémiologique intégrée - prolongation (PASÉI 2 – prolongation)
15. Évaluation conjointe de «l’intervention de Viva Rio dans la zone de bel air» en Haïti – appuyée par l’ACDI, le MAECI/GTSR et NCA
16. Observatoire du Sahara et du Sahel (OSS)
MGPB 17. Evaluation of the Micronutrient Initiative
PWCB 18. Strengthening Access to Justice through Legal Sector Development
19. International Legislative Audit Assistance Program

Review of Evidence of the Effectiveness of CIDA's Grants & Contributions Gs&Cs 2005–2011 UKRAINE COUNTRY PROGRAM EVALUATION
© CIDA/JOSHUA KRAEMER
Review of Evidence of the Effectiveness of CIDA's Gs&Cs 2005-2011
For more information on this
evaluation, visit our website or scan
the QR code:
www.cida.gc.ca/evaluations-e
This quantitative and qualitative review of the grants and contributions (Gs&Cs) of the Canadian International Development Agency (CIDA) systematically reviewed and analyzed 41 major evaluations completed (between 2005 and 2011) across CIDA’s three main programming channels: bilateral, multilateral, and partnership. The review rated CIDA’s effectiveness along the dimensions of relevance, achievement of objectives, and sustainability, and also explored initiatives undertaken by CIDA over the past few years to improve its effectiveness.
Why conduct this evaluation?
The Treasury Board of Canada is responsible for approving and reviewing the continuation of transfer-payment programs. This review supported the renewal of CIDA’s terms and conditions.
What did the evaluation find? CIDA’s contributions are relevant and achieve objectives, but sustainability is more of a challenge.
Relevance: All programming was identified as being relevant. Key factors that contribute to relevance are ensuring quality at entry (that is, a beneficiary needs assessment, due diligence, and program design based on analysis and coordination with other donors), alignment with Paris Declaration principles, and continued due diligence during implementation.
Achieving results: CIDA programs and projects generally achieved their objectives. Overall, program objectives were most likely to be achieved when there were good, planned policy engagement and dialogue with partners, and clear statements of expected results and the risks to be managed. Complementarity among CIDA’s funding channels and the use of a mix of funding mechanisms are other factors that contributed to the achievement of objectives.
Sustainability: Sustainability was assessed as being good to average overall, with somewhat lower ratings than the achievement or relevance of objectives. Sustainability was more likely to be achieved if it was planned, and adequate financial and human resources were invested. Sustainability was enhanced by local ownership, commitment, and capacity, and by trust based on understanding needs, capacities, perspectives, mandates, and context.
Crosscutting issues: The integration of crosscutting issues— gender, environment, and governance—was uneven across CIDA programs.
What challenges does CIDA face?
 Ensuring quality at entry: designing projects that relate to local needs and the overall country strategy, but are not over- ambitious.
 Working effectively with new aid modalities: designing programming approaches in collaboration with recipient country governments, institutions, civil society, and other donors.
 Managing decentralization: ensuring the right mix of sectoral and technical expertise at the field level and delegating appropriate authority.
 Using monitoring and evaluation as tools to learn and improve across all channels: systematically extracting and disseminating lessons learned. Most evaluations reported results at the output and outcome levels; few reported results at the impact (or ultimate outcome) level.
Where is CIDA headed?
 Instead of recommendations and a management response, this report examines CIDA’s ongoing efforts to improve effectiveness.
 CIDA has developed a more focused, results-based policy framework, which has been translated into renewed planning processes. CIDA focuses on one strategic outcome: the reduction in poverty for those living in countries where CIDA engages in international development.
 CIDA’s Aid Effectiveness Action Plan guides its work, and includes accountability and monitoring guidelines.
 CIDA has undertaken a number of initiatives to improve the efficiency of its aid, including improving its long-term planning processes and mechanisms for managing risk while it modernizes its business processes. 13

Country Program Evaluation Ukraine 2004-2009 © CIDA/CAROL HART
Ukraine Country Program Evaluation 2004-2009
the QR code:
In 2004, CIDA responded to Ukraine’s renewed efforts for economic and political reform, following the country’s Orange Revolution. The program focused 95 percent of its funding to support development in two areas reflecting the shared goals of Canada and Ukraine: sustainable economic growth and democratic development. By 2009, CIDA had invested more than $99 million in programs to benefit Ukraine’s 45 million people, making Canada Ukraine’s fourth- largest provider of official development assistance since 2004.
This evaluation assessed the performance and the results achieved by CIDA’s Ukraine Program between 2004 and 2009. The evaluation focused on a sample of 20 projects, representing 57 percent of total program investments.
What did the evaluation find? The evaluation found CIDA’s Ukraine Program to be highly effective in achieving measurable results and that CIDA’s projects in Ukraine had become models at CIDA, in Ukraine, and internationally.
Relevance: The program’s high relevance to Ukrainian and Canadian priorities reflected its skillful and responsive design. Economic development agricultural projects, for example, achieved such high levels of relevance that the Government of Ukraine requested they continue.
Effectiveness: Ukraine frequently followed through to establish institutions, policies, and laws as a result of CIDA project support, for example in the area of civil service and judicial reform, signalling the projects’ immediate and promising long-term effectiveness.
Sustainability: CIDA stood out among Ukraine’s donors for flexible project design and durations, to allow projects to consolidate gains and build a basis for sustainable results.
Crosscutting themes: Programming efforts in the crosscutting areas of gender equality and the environment and in youth did not achieve expected results. Entrenched social attitudes in Ukraine constrained gains in gender equality. Minimal attention to programming in environment and youth yielded only modest results.
Coherence: The program’s design promoted coordination (termed coherence) with other CIDA and donor projects. Coherence was greater in the portfolio of economic growth projects, but there were more limited opportunities to build coherence among democratic development projects. The program needs to find more effective entry points to achieve gender equality results, and to devote more sustained attention to environmental sustainability.
Efficiency: Conscientious management of human and financial resources created efficiencies. For example, some projects optimized currency exchange and incorporated local knowledge and skills.
Management principles: The program implemented the 2005 Paris Declaration on Aid Effectiveness management principles and realized stakeholder ownership, but only at the project level, in the absence of a Ukrainian national development plan.
Management performance: The program implemented results- based management principles, although frequent staff turnover led to some inconsistencies in reporting.
How is the Ukraine Program responding?
 By continuing and by consolidating programming in the areas of sustainable economic growth and democratic development;
 By facilitating greater coherence between national and regional level projects;
 By balancing directive and responsive programming;
 By establishing new partnerships and sharing information;
 By improving gender equality and environmental integration; and
 By enhancing monitoring and evaluation of projects and programming context, in part through decentralization of operations and management. 14

Regional Program Evaluation Inter-American 2004–2010 UKRAINE COUNTR
Y
What is next for CIDA’s Inter-American Program?
PROGRAM EVALUATION
© CIDA/ROGER LEMOYNE
Inter-American Regional Country Program Evaluation 2004-2010
the QR code:
The Inter-American Program of the Canadian International Development Agency (CIDA) promotes development in Latin American and Caribbean countries by focusing on regional challenges, such as economic growth and disease. Between 2004 and 2010, it disbursed $242.5 million.
The evaluation assessed the program’s performance and results achieved between 2004 and 2010. It sampled two thirds (66 percent) of program expenditures, totaling $145.5 million.
What did the evaluation find? The evaluation found a program that brought tangible, long- term development benefits to the region, including the elimination of rubella in the Americas.
Relevance: The program’s aims proved highly relevant to the Millennium Development Goals for 2015 agreed to by United Nations member nations and their allied institutions. More attention, however, to maintaining a strategic focus for the program’s duration will help it achieve greater relevance to the region’s emerging needs. Nonetheless, the program’s focus on health, governance, and the private sector did reflect key regional concerns.
Effectiveness: Concrete benefits in the areas of health and private-sector development demonstrated the program’s effectiveness. The results of various projects are visible both within national governments and in the private sector as stakeholders chose to deploy the skills, tools, strategies, and policies that the program nurtured. Such results did not emerge as clearly in areas related to governance.
Sustainability: The program’s two key partners are the Organization of American States and the Pan American Health Organization. Canada’s participation in these international organizations helps to promote the program’s goals in the long term.
Crosscutting themes: The program met only moderate success promoting two of CIDA’s crosscutting priorities: gender equality and the environment. The program did not monitor the systematic implementation of recommendations from CIDA’s gender and environmental specialists. A few projects, however, made gains in improving gender equality and environmental sustainability.
Coherence: The program is not well known inside CIDA, and connections to other parts of CIDA were established ad hoc. The program’s coordination across various Canadian departments has been functional but underwhelming. One constraint is that the program is often the major Canadian funder, but not the designated government lead on partner interactions.
Efficiency: Health sector efficiency was assessed to be satisfactory, but evaluators were not able to score governance and private sector areas, given the difficulty of assessing institutional effectiveness within the limits of this evaluation.
Management principles: The program performed well in promoting stakeholder ownership as outlined in the 2005 Paris Declaration on Aid Effectiveness. Canada is championing aid effectiveness principles by providing flexible funding and by investing in institutional strengthening mechanisms with its key strategic partners. CIDA’s somewhat guarded approach to dealing with stakeholders while developing its 2010–2015 Regional Development Programming Framework is contrary to the principles of the Paris Declaration.
Performance Management: The program belatedly applied a management framework and the projects’ use of result-based management tools was uneven.
 Strengthen information and coordination with other CIDA and Canadian programs in the region.
 Better integrate and implement gender equality and environmental sustainability goals.
 Invest in regional and subregional programming through new ideas, new players, and new funding mechanisms.
 Invest in performance-management tools and continue to provide results-based management support to partner organizations.
 Spell out performance-management expectations to partner organizations and mitigate partner weaknesses during up- front negotiations of funding agreements. 15

Country Program Evaluation Peru 2005-2010 For more information on this
the QR code:
© CIDA/ JOSHUA KRAEMER
Peru Country Program Evaluation 2005-2010
In the past forty years, Canadian development assistance in Peru has evolved from a diversified approach to a more focused approach, targeting key sectors in support of the country's national poverty-reduction efforts. Between fiscal years 2005- 2006 and 2009-2010, CIDA disbursed a total of $172 million to Peru.
Why Conduct this Evaluation?
The purpose of this evaluation was to assess the performance and results achieved through CIDA’s three delivery channels (geographic, partnerships with Canadians and multilateral and global programs) in Peru between 2005 and 2010. The evaluation focused on a sample of 27 projects in Peru, with a total disbursement of $88 million. CIDA’s Peru Program was rated as highly satisfactory. The evaluation found that CIDA’s decentralized process contributed to this result
What did the evaluation find?
Education: CIDA’s Peru program made significant progress in education despite the continuing weakness of public institutions, particularly in rural areas. CIDA helped improve the quality of, and access to, education, and strengthened management and administrative capacities. For example, a rural education model developed through the program led to impressive learning improvements in reading and writing for primary school children. However, the results were less satisfactory for early childhood development.
Governance: The program contributed to increased respect for democratic principles, as well as improved accountability and transparency in government and in strategic sectors. For example, the Ombudsman’s office was strengthened and 28 regional offices developed to help citizens access services. There is still room for improvement in CIDA’s governance programming, particularly with respect to policy dialogue and private sector development.
Private Sector Development: The program contributed to improved management of the mining and hydrocarbon industries in Peru and increased transparency. It also explored mechanisms for conflict resolution among stakeholders.
Gender Equality: The program succeeded in reducing gender inequality in education and improving women's capacity for participation in local government decision making and planning. Improvements are still needed in reducing violence against women and increasing their participation in the extractive sector.
Humanitarian Assistance: The program was effective in providing services to the most vulnerable people. It contributed to improved institutional capacities of community organizations and several regional authorities and of the management of municipal water and sanitation services. It also helped create a culture of preparedness and risk prevention. However, issues such as revenue and tax distribution, as well as royalty-sharing arrangements among private companies, the national government, and regional authorities are still problems.
Environment: As a result of CIDA’s strategic environmental integration planning, the program helped improve regulatory frameworks in the public and extractive sectors, supported advocacy campaigns on environmental protection, and promoted environmental sustainability. However, the environment was not adequately addressed in the education sector, and the entire program would benefit from further steps to enhance environmental considerations. There is still room for improvement in CIDA’s governance programming in Peru, particularly with respect to policy dialogue and private sector development
What is next for CIDA’s Peru Program?
The Program plans to:
 pursue interventions that strengthen regional and local governments
 develop strategies with other stakeholders to sustain the results and benefits of its interventions in decentralization and governance
 enhance its results-oriented performance management tools and methods
16

Country Program Evaluation Honduras 2002–2010 UKRAINE COUNTRY PROGRAM EVALUATION
© CIDA/BRIAN ATKINSON
Honduras Country Program Evaluation 2002-2010
the QR code:
In 2002, CIDA identified Honduras as a country prioritized for Canadian official development assistance. By 2010, CIDA’s Honduras Program had disbursed $134.5 million dollars, fostering Honduran ownership of development initiatives at the local, regional, and central levels.
The purpose of this evaluation is to contribute to program improvement and to assess both the program’s performance and delivery mechanisms between 2002 and 2010. The evaluation sampled 26 projects, representing $40 million dollars of total funding.
What did the evaluation find? The evaluation found a highly relevant and effective program that implemented successful health and food security projects, contributing to CIDA and Honduran aims for poverty reduction.
Relevance—The high relevance of the program’s projects to Honduran realities stemmed from its focus on local and regional projects and partnerships and from the strong alignment between CIDA and Honduran priorities, particularly on the issue of poverty reduction.
Effectiveness—Two key sectors, food security, which includes natural resources management, agriculture, forestry, water and sanitation, and watershed management, and health, successfully achieved their development goals, making them effective, especially in combating Chagas disease and in helping to prevent the spread of sexually transmitted diseases among Honduran youth. Honduran targets for basic education, however, were below expectations.
Sustainability—The program successfully implemented initiatives designed to persist after their completion and to sustain the benefits of CIDA’s investment. The inclusion of local stakeholders in CIDA projects assures a legacy of trained and experienced community leaders in Honduras who can use the experience they gained under CIDA’s auspices in other arenas also contributing to sustaining CIDA’s investment.
Crosscutting themes—The program’s activities in CIDA’s areas of priority—gender equality, governance, and environment— termed crosscutting themes, are difficult to assess globally due to the variety of funding mechanisms used to deliver them. They appeared, however, satisfactory.
Coherence—Lack of formal discussions about strategy at the program level resulted in lower levels of coordination—termed coherence—among CIDA branches and Canadian stakeholders. However, the better coherence of the local and regional projects enabled their continuity during the 2009-political crisis in Honduras. More discussion and consultation among Canadian stakeholders can help improve the program’s lower levels of coherence.
Efficiency—The program operated efficiently overall; local staff created added efficiencies by consolidating their institutional knowledge.
Management principles—The program’s mix of local, regional, and central level projects allowed it to promote Honduran ownership in delivering the aid, in line with the Paris Declaration on Aid Effectiveness, when circumstances in Honduras prevented strong ownership by the central government.
Performance management—Developing and applying performance measurement and risk management frameworks remains an ongoing process.
What is next for the Honduras Program?
 Continued deployment of multiple mechanisms for aid delivery at local, regional, and central levels
 Increased attention to joint process, such as workshops and taskforces that will promote efficient cooperation between Canadian stakeholders
 Systematic incorporation into CIDA’s knowledge base of the local experience and knowledge generated by the program’s Honduran professionals
 Systematic annual planning and monitoring of activities, such as policy dialogue and coordination among donors
17

Humanitarian and Development Effectiveness
Review of the World Food Programme WFP 2006-2011 © CIDA/JEAN-FRANCOIS LEBLANC
Humanitarian and Development Effectiveness Review of WFP 2006-2011
the QR code:
The World Food Programme (WFP) is both a humanitarian and development United Nations agency. It is the world’s largest humanitarian agency fighting hunger worldwide. In emergencies, it distributes food where it is needed to save the lives of victims of war, civil conflict and natural disasters. After the cause of the emergency passes, it focuses on development, using food to help communities rebuild their lives.
Why Conduct this Review?
The purpose of this review is to provide an independent, evidence-based assessment of the humanitarian and development (H&D) effectiveness of WFP for use by Canada in its decision-making regarding resourcing WFP. The information can also be useful for other stakeholders. The review was based on the content of 52 WFP evaluations published between 2006 and 2011. WFP programs are highly relevant to their contexts.
 WFP programs are highly relevant to the needs of the target groups and developing country governments. Factors contributing to success in this area include the quality of WFP’s needs assessments and the nature of the arrangements of the partnerships created.
 WFP is effective in achieving its HA&D objectives and expected results. A key factor contributing to WFP effectiveness is its strong logistics capacity. However, this effectiveness is weakened at times by program interruptions due to commodity shortages or unpredictability of donor funding.
 Findings on sustainability of the benefits/results of WFP programming represent an area for improvement for the organization. Some programs had weak exit strategies and did not build sustainability into program design.
 Efficiency represents another challenge for WFP in some areas. Program interruptions were identified as hindering success in this area.
 WFP has not been as effective in addressing the cross cutting theme of gender equality. Successes, however, were noted in the areas of the provision of sex-disaggregated data and some improvements for women and girls.
 The cross cutting theme of environmental sustainability was not adequately addressed in the evaluation reports reviewed, which prevented the review from identifying any results in this area.
 While WFP is effective in its use of evaluations, monitoring and reporting on results is a challenge. Factors that detracted from success in this area include insufficient capacity for monitoring and inadequate performance frameworks for programming. Program interruptions hindered success in the area of efficiency.
What is next for CIDA?
 Encourage WFP to resolve the program interruptions.
 Identify the sustainability of WFP programming as a priority strategy for its engagement with WFP.
 Emphasize the need for WFP to improve its effectiveness in promoting gender equality.
 Underscore to WFP the importance of following up on its commitment to address environmental issues.
 Emphasize to WFP the need to develop performance frameworks that adequately reflect the expected results of WFP programming.
18

Evaluation of CIDA’s Humanitarian Assistance Humanitarian Assistance 2005–2011 © CIDA/NICK WESTOVER
Evaluation of CIDA’s Humanitarian Assistance 2005-2011
the QR code:
CIDA’s humanitarian assistance aims to save lives, alleviate suffering, and maintain the dignity of those affected by conflicts and natural disasters by providing appropriate, timely, and effective assistance.
Between 2005 and 2011, CIDA provided more than $2.7 billion in humanitarian assistance. This evaluation provides an oversight of this spending and examines how to enhance the relevance, design, delivery, and performance of our assistance. CIDA’s contributions help save lives, alleviate suffering, and maintain human dignity
 Relevant to Beneficiaries: CIDA has the appropriate tools to collect and analyze needs-based information to ensure it is choosing relevant proposals by its partners.
 Global Results: Canada has played an important role in the international humanitarian community, and continues to contribute significantly to global coordination and capacity while promoting results- based management and gender equality.
 Country-Level Results: CIDA’s contributions are largely effective in meeting CIDA’s objectives, with results achieved for all expected outcomes. Concrete results for beneficiaries included improving access to food and non-food items, re-establishing livelihoods, reducing vulnerability, and providing protection.
 Expertise and Design: CIDA’s humanitarian assistance expertise is robust and relevant, and is concentrated in Headquarters in the International Humanitarian Assistance Directorate. CIDA field staff would benefit from better guidance on their mandates and coordination responsibilities vis-à-vis humanitarian issues.
 Monitoring and Evaluation: More systematic monitoring and evaluation would help CIDA to better capture, integrate and share lessons from its humanitarian assistance program.
 Efficiency: In the past six years, CIDA decreased the percentage of humanitarian funds spent on management from an already efficient 1 percent.
 Matching Funds: There are no public criteria for determining when to launch a matching fund. The use of matching fund monies could be more transparent.
 Communication: CIDA regularly informs the media about its humanitarian assistance responses. CIDA is missing opportunities to communicate the achievements of its humanitarian programming to the public in both Canada and partner countries.
What Does the Evaluation Recommend?
 Develop a whole-of-agency humanitarian assistance strategy.
 Develop a systematic, integrated approach to supporting (a) prevention and risk reduction, and (b) recovery and transition to development.
 Intensify efforts to improve the timeliness of humanitarian response decision-making.
 Review the use of matching funds.
 Improve accountability and reporting through the use of monitoring and evaluation to identify lessons learned and to ensure adequate dissemination and implementation.
 Improve the information available to the Canadian public about CIDA’s humanitarian assistance activities. 19

Development
Effectiveness Review of the United Nations evelopment Programme UNDP 2005-2011 D
the QR code:
Development Effectiveness Review of the UNDP 2005-2011
The United Nations Development Programme (UNDP) is the United Nations’ global development network, supporting national processes to accelerate the progress of human development, with a view to eradicate poverty.
The purpose of this review is to provide an independent, evidence-based assessment of the development effectiveness of UNDP. The review synthesized 55 evaluations that concentrated on UNDP programming between 2005 and 2011. Reviewed evaluations report that UND programs are relevant to the development context in which they operate.
 The reviewed evaluations report that UNDP programs are relevant to the development context in which they operate. Factors contributing to success in this area include effective consultation with stakeholders to build consensus on needs and solutions.
 The reviewed evaluations also indicate that most UNDP programs achieve their development objectives and expected results, although weaknesses remain in some areas. Success was hindered by dispersion of UNDP programming across too many projects, too wide a geographic area or too many institutions.
 The sustainability of program results and benefits represents a significant challenge to UNDP’s development effectiveness. Some programs lacked an explicit phase-out strategy and did not integrate sustainability considerations into program design.
 Efficiency is another area requiring improvement, with many reviewed evaluations reporting weaknesses in program efficiency. The complexity of UNDP’s project administration systems and procedures hinders greater efficiency.
 UNDP is generally effective in supporting gender equality. However, a lack of systematic gender analysis hindered wider success in this area. In some
programs, UNDP mainstreamed gender equality and increased women’s participation in governance structures.
 UNDP effectively supports environmental sustainability. Factors contributing to success in this area include UNDP’s effective advocacy for environmental issues and its willingness to invest in environmental research methods and tools.
 Weaknesses were reported in decentralized systems for evaluation, monitoring and results-based management. A tendency to focus on outputs rather than outcomes limits success in this area. However, UNDP’s Evaluation Office produces quality evaluations, including detailed management responses. Reviewed evaluations also report that UNDP’s program efficiency requires improvement.
What is next for CIDA?
 Emphasize the need for some UNDP programs to focus on fewer projects within a given country.
 Promote the systematic use of gender analysis during UNDP’s program design.
 Designate improvements in sustainability of benefits as a priority strategy for engagement with UNDP.
 Engage with UNDP to improve program efficiency at the country level.
 Encourage UNDP to strengthen decentralized systems for monitoring, evaluation, and results-based management. 20

f
Nepal Peace Building rom Below (PBB) Project NEPAL PBB 2008–2012 Nepal Peacebuilding from Below Project Evaluation 2008-2012 © CIDA/PAT MORROW
Why evaluate PBB?
PBB was a $3.05-million, three-year project that ended in 2012. It aimed to bring development opportunities to communities in conflict zones that addressed the root causes of conflicts and promoted reconciliation. Rural Reconstruction Nepal (RRN)—one of Nepal’s largest domestic non-governmental organizations (NGOs)— implemented the PBB, working through the intermediary of 150 community-based people’s organizations (CPBOs) and other stakeholders located in two hilly districts of the Eastern Development Region of Nepal.
What was the object and scope of the evaluation?
CIDA commissioned the summative evaluation for general accountability purposes and to gauge the merits of funding similar projects in the future.
Unique programming element
PBB represents an innovation for CIDA in that it was implemented by a local executing agency (RRN) rather than a Canadian one.
What were the key conclusions and recommendations?
 CPBOs realized significant capacity gains in areas such as participatory planning and needs prioritization, account keeping, dispute resolution, gender equality and social-inclusion awareness, and access to government resources.
 Gender equality and social-inclusion sensitization was beginning to create shifts in ancient attitudes toward women, and toward the Dalits and Janajati castes.
 PBB’s systems and procedures, staffing structure and monitoring provisions were satisfactory overall.
 Many of PBB’s results will be sustainable partly because the CBPOs developed the institutional and financial capacity to continue their community development using best practices. Less-sustainable results, however, stemmed from the peace-building initiatives, still sensitive to tensions and the effects of competing political forces.
 As a demand-driven project, PBB was highly relevant to needs at the community level.
What key lessons were learned?
 The steepness of PBB’s learning curve suggests that a successor project would benefit from more time spent at the inception phase, and that the RRN, CIDA, and the Canadian Cooperation Office would benefit by discussing performance and financial reporting expectations.
 Successor projects will need performance-based, realistic, and flexible targets because the operating environment in Nepal is very challenging and development itself is iterative in nature.
 The PBB model was well received by the Ministry of Peace and Reconstruction, and should be shared with other donors and NGOs. This could provide practical feedback and could create interest in partnerships for the replication of this model. 21

Sri Lanka Local
Initiatives
For Tomorrow – Phase 2
(LIFT2)
Sri Lanka LIFT2
2008–2013
Sri Lanka Local Initiatives for Tomorrow Phase 2 Evaluation 2008-2013
© CIDA/STEPHANIE COLVEY
Why evaluate Local Initiatives for Tomorrow – Phase 2 (LIFT2)?
The $4.65-million, five-year LIFT2 project aimed to transform communities and livelihoods in 150 of the poorest and most socially marginalized villages in Sri Lanka’s war-affected areas of Jaffna, Polonnaruwa ,and Batticaloa by building the capacities of local initiative groups (LIGs) and 190 of their respective community- based organizations.
This midterm evaluation assessed the program’s first two cycles with a view to informing the project’s final 18-month cycle. In particular, the evaluation assessed the design and implementation plan of the final cycle for its appropriateness to the new political and economic landscape in Sri Lanka. The evaluation will also inform future CIDA programming in Sri Lanka.
 The project fulfilled CIDA’s special emphasis on the country’s economic growth, which is aligned with the Government of Sri Lanka’s priorities for poverty reduction, and it related well to communities’ needs. However, several of its components needed better linkages.
 LIFT2 reached targeted villages and beneficiaries. Village selection, community mobilization, and group- formation methods were impressive and effective. District- and divisional-level governments were highly involved, ensuring responsiveness and buy-in.
 The focus on livelihoods has been appropriate, given the socio-economic situation of the targeted villages. In particular, the project’s focus on savings and credit, small-community infrastructure, and income- generating activities (IGAs) contributed to economic empowerment. The IGAs, however, will need nurturing and market links to succeed.
 LIFT2’s initial strategy on disaster and environmental change response (DER) was unclear, but in the past year, it did make some progress toward attaining the objectives. More funding for training on DER would help. The project appropriately and effectively adapted to changes on the ground because it benefited from the linkages it promoted among government agencies at the district, division, and local levels.
 For long-term growth, the evaluation recommends initiatives for economic growth, and for women and the poor.
 Eighteen months is not a sufficient time frame in which to expect the major transformations the project envisioned, even with a midterm evaluation to inform the final cycle.
 The project could have taken more advantage of the LIGs’ leadership capacities.
 Building “value chains” is a sound livelihood strategy that requires constant up-to-date research, dedicated expertise, and contact building and nurturing.
 The project missed opportunities to engage the local non-governmental organizations that could have contributed to the LIGs and helped advocate for and communicate with the LIGs’ districts or divisions. 22

The Ukraine Civil Service
Human Resources
anagement Reform Project
kraine CS HRM Project
2006
2012
-
–
© CIDA/ANDRII SKAKODUB
M
U
Ukraine CS-HRM Project Evaluation 2006-2012
Why evaluate the Ukraine Civil Service Human Resources Management project?
The Ukraine Civil Service Human Resources Management (UCS- HRM) reform project was designed and implemented by the Canadian Bureau for International Education. The main goal of the UCS-HRM project was to promote transparent and accountable governance in Ukraine through targeted reforms of the central government’s human resources management system based on European standards established by Support for Improvement in Governance and Management (SIGMA) and the Organisation for Economic Co-operation and Development (OECD). The project’s design and implementation ran from 2004 to 2012, with a budget of approximately $5.8 million.
This summative evaluation documented the project’s achieved results, reported on its progress, and assessed its likely outcomes with a view to informing CIDA management about the project’s challenges and opportunities, and lessons learned.
 The UCS-HRM project was a well-designed and well- managed intervention despite its implementation in a relatively volatile environment.
 The project made substantial progress toward its targets, and considering its current rate of progress, will likely achieve all of its targets by the end of its extended life: June 30, 2012. In particular, the project will mostly likely succeed in making a strong contribution to its core objective: a competent and professional civil service operating within an improved HRM governance framework and with respect for human rights and democratic principles.
 There is a risk that gains will not be fully realized by using local resources alone, and thus not be sustainable in the long term. Further funding is likely to be necessary to support the widespread adoption of HRM tools.
 Indicators to measure project performance range from the very simple and specific to the very broad and inherently difficult to measure. The project should put in place a mechanism monitor and report on relevant key performance indicators.
 There are advantages to adopting a design-and- implement approach to the procurement of projects of this kind. For example, the lengthened design period allowed for mutual trust and confidence to develop.
 A two-year design phase resulted in a strategic approach in which both the project team and the Ukraine government aligned their activities and provided a firm foundation for more effective project design and execution.
 The project’s clear structure of intended outcomes, outputs, and activities facilitated project management and reporting, and helped to define stakeholder roles and responsibilities.
 In a volatile political environment, flexibility is essential to seize opportunities and manage risks. For example, the project went further than originally planned when it grasped an opportunity to extend the leadership competencies it had fostered into a broader approach for managing the whole senior civil service.
 Finally, simple project-governance structures can be appropriate for tightly focused projects with a limited number of immediate stakeholders, but when wider reforms are contemplated, a more complex governance structure involving more key actors may be required in order to build a broader consensus. 23

-
affected Areas of
Agricultural Development in
)
2009
Project
–
CIDA/
Microfinance Component of the
(ADMAC
Cambodia
ADMAC
2006
©
Mine
ADMAC Project Evaluation 2006-2009
ROGER LEMOYNE
Why evaluate the Microfinance Component of the Agricultural Development in Mine-affected Areas of Cambodia Project?
Between 2006 and 2009, CIDA disbursed $5 million to the Microfinance Component of the Agricultural Development in Mine-affected Areas of Cambodia project through the Ministry of Agriculture, Forestry and Fisheries Project Support Unit.
Overall, the project installed 34 agricultural cooperatives in Cambodia, benefiting nearly 5,000 farmers by focusing on strengthened production, better access to financing services, and the provision of training and technical support. Two key components of the project provided specialized direction: the production start-up program component supplied technical support and a revolving loan fund and the agricultural microfinance component supported the establishment of a series of community savings and credit groups.
The evaluation assessed the performance of CIDA- funded agricultural cooperatives, which were part of the project and which specialized in microfinance services.
Unique programming elements
This was the first project to link agricultural development with mine mitigation.
 The Ministry of Agriculture, Forestry and Fisheries and the provincial departments of agriculture in Cambodia did an excellent job establishing agricultural cooperatives, and their subsequent sound performance showcased the project’s effective approaches and support mechanisms.
 Performance across the agricultural cooperatives varied considerably. Some of the weaker cooperatives have yet to attain sustainability, and will need outside support in the short term.
 Further support networks will be needed as the agricultural cooperatives begin to stabilize operations and grow. The evaluation highlighted a series of recommendations:
1. The Ministry of Agriculture, Forestry and Fisheries and the provincial departments of agriculture should develop and implement a plan for strengthening the capacity of the cooperatives over the next two years.
2. The Ministry of Agriculture, Forestry and Fisheries and CIDA should work together to identify methods to leverage other donor funding to support the agricultural cooperatives.
3. CIDA should promote capacity development activities for the agricultural cooperatives, possibly by establishing connections with CIDA’s other partners.
 Microfinance programs take substantial time and resources to implement and develop.
 Cooperatives are one method for farmers to improve their economic prospects, but participants should be aware of the risks, implications, and benefits.
 Establishing a cooperative requires adequate financing before becoming operational. 24

Regenerating the Murad
Khane District of Kabul
Project (RMKD)
Afghanistan RMKD
2008 2011
–
Afghanistan Regenerating the Murad Khane District of Kabul Project Evaluation
2008 2011 - © CIDA/ ROGER LEMOYNE
Why evaluate the Regenerating the Murad Khane District of Kabul Project?
Between 2008 and 2011, CIDA participated in the Turquoise Mountain program in Afghanistan with a $7-million dollar project: Regenerating the Murad Khane District of Kabul.
Since 2006, the Turquoise Mountain program has worked to regenerate Afghanistan’s traditional crafts and historic areas with the twin goals of creating jobs and skills, and renewing a sense of national identity. The Regenerating the Murad Khane District of Kabul project sought to help improve the quality of life and living conditions in the district, while also developing Afghanistan’s crafts sector.
CIDA commissioned the evaluation of this project for accountability purposes and to gauge the project’s effectiveness at achieving its aims.
Regenerating the Murad Khane District of Kabul is a distinctive project because it focused on cultural issues and concentrated on a specific location, which enhanced its impact and efficiency.
 CIDA implemented the project according to plan with effective results. It set a multidimensional development agenda that went well beyond the founding idea. This project supported the economic, social, and cultural regeneration of the district of Murad Khane through a variety of measures aimed at raising the community’s standard of living, enhancing its economy, and strengthening its traditions and identity.
 The project has obvious relevance to the community’s needs, and it contributed to its economic improvement and to a rising standard of living for its residents.
 Overall, the Turquoise Mountain program did a good job of managing its staff and its liaisons with its long list of donors.
 The project focused on maintaining cooperative relationships with local stakeholders, and it won the backing of several government units and cultural institutions.
 Turquoise Mountain adopted strong policies relating to CIDA’s priority areas (known as crosscutting themes). Attention to gender issues, especially in the community development component, was successful. Environmental initiatives to remove garbage and install waste, water, and drainage systems all had positive results.
 Building and maintaining constructive ties with a strong community of leaders helped the project’s smooth implementation and was a key factor in its success.
 All donor-funded interventions should define from the outset an exit strategy that plans for long-term sustainability and prepares to meet the multiple contingencies typical of an environment such as that found in Afghanistan.
 Rather than multiple project sites, concentration on a single geographic location allowed the project to achieve—and exceed—objectives. 25

PASÉi 2
2003–2011
Evaluation of the Integrated
Epidemiological Surveillance–
Phase II Extension
2003–2011
PASÉi 2 Extension Project Evaluation 2003-2011 © CIDA/SAMUEL GERVAIS
Why evaluate the PASÉi 2 Extension
Between 2003 and 2011, CIDA invested a total of $18 million dollars in the Integrated Epidemiological Surveillance–Phase II project (PASÉi 2). Executed by the Centre for International Cooperation in Health and Development, it helped the ministries of health in Benin, Burkina Faso, Guinea, Mali, and Niger implement an early- warning system for epidemiological surveillance (an aspect of disease control). The project’s successes led a 2007 CIDA evaluation to recommend extending it in time to consolidate gains and ensure their viability in national institutions.
The extension project helped participant countries build disease-control capacity for the long term by supporting training programs, supplying equipment, and aiding coordination and harmonization activities. It also adopted a regional, rather than a national, focus—responding to the reality that diseases do not recognize political borders—to promote the standardization of practices to allow regional cooperation.
The 2011 evaluation of the PASÉi 2 Extension Project assessed its achievements in the participant countries and measured them against the extension project’s aims. In particular, this evaluation assessed the project’s incorporation of the recommendations from the 2007 evaluation.
 The extension project’s positive results in consolidating PASÉi 2’s gains demonstrated the benefits of longer- term planning.
 CIDA needs to ensure that implementation plans and results-based management tools meet CIDA’s standards, even for unsolicited proposals.
 The PASÉi 2 project demonstrated the benefits of maintaining a tight focus upon a few objectives, rather
than attempting to exhaust every dimension at the risk of overreaching itself.
 A study seeking a fuller description of the roles and responsibilities for intervention epidemiology at all levels of the health system will support training harmonization.
 CIDA needs to liaise with the World Bank about West African health projects in order to seek greater harmony between its future projects and those of the World Bank and to discuss the possibility of collaboration. 26

Evaluation of the VIVA RIO
project in Port-au-Prince,
Haiti
VIVA RIO
2006–2010
Viva Rio Project Evaluation 2006-2010
Why evaluate the Viva Rio project?
Since 2002, Viva Rio, a Brazilian non-governmental organization (NGO), has helped regenerate the embattled and poor neighborhood of Bel Air in Port-au- Prince, Haiti.
The Canadian International Development Agency (CIDA), Canada’s Department of Foreign Affairs and International Trade, and the Norwegian Church Aid NGO contributed independent, but interlocking, projects totaling $11 million (excluding emergency aid) to the $12-million Viva Rio initiative.
The funding partners commissioned the present evaluation, which covers four years (2006–2010). It takes stock of the program’s lessons learned, and assesses the partners’ decision-making and funding processes.
The 2010 earthquake in Haiti took a heavy toll on Bel Air. As a result, Viva Rio suspended planned initiatives and instead met urgent needs for humanitarian aid.
The evaluation revealed an effective program that achieved concrete benefits for the neighborhood and its residents under very adverse conditions.
However, the evaluation made several recommendations in order to assure the program’s longer-term success. The evaluation recommended that Viva Rio:
 strengthen its organizational and management structures;
 coordinate with projects offered by other members of the donor community; and
 develop better gender-equality programming.
The evaluation also recommended that the funding partners:
 coordinate to uphold management and organizational standards for Viva Rio, and share information more effectively.
 Innovation and responsiveness characterized Viva Rio’s interventions, and by nurturing this, the funding partners helped create the basis for success.
 The immediate and visible results achieved by the program should be bolstered by long-term planning.
 Long-term results will most likely materialize if the program can resume its planned interventions under adequate conditions and within a ten-year, rather than a five-year, time frame.
 Alliances and coordination with other donor groups will help Viva Rio avoid overextending itself but still offer comprehensive and sustained aid to Bel Air.
 Sociological and anthropological research undertaken as part of the Bel Air initiatives fine-tuned the projects and stands as a shining example.
 South-South leadership initiated by Viva Rio introduced culturally and geographically relevant solutions to Bel Air and helped develop local expertise. 27

Sahara and
Observatory
–
Sahel
Evaluation
OSS
2005 2010
© CIDA/PIERRE ST JACQUES
-
Sahara and Sahel Observatory Evaluation-2005-2010
2005-2010PROJECT
Why evaluate the Sahara and Sahel Observatory?
Between 2005 and 2010, CIDA participated in the Sahara and Sahel Observatory (OSS) programming, disbursing $3 million through its West Africa Program and its West and Central Africa Regional Program to support the OSS’s role as a neutral environmental forum promoting sustainable practices relevant to a region prone to drought and desertification.
CIDA aims to build the lessons learned from this evaluation into future programming for the OSS and to share the evaluation results with OSS partners.
Considerations for a new OSS proposal for CIDA funding makes this evaluation very timely.
The evaluation returned general recommendations for the OSS and specific recommendations for CIDA’s internal programming.
General recommendations for the OSS:
 The OSS can consolidate its areas of expertise, build up its observatory functions, and develop expertise related to climate change.
 The environmental monitoring and evaluation system of the OSS can be improved by integrating data from external and multilateral evaluation systems.
 The OSS can maintain its light management structure and mandate while pursuing its efforts to increase its financial contributions from countries and organizations in the region.
 The OSS needs to develop the partnerships suggested by the board and the strategic
orientation committee, and to pursue ones with Canadian organizations.
 The OSS can improve communication with its users by linking its website to those of partner countries, particularly with national focal points.
 The OSS needs to finalize and implement its Gender Equality Action Plan.
Recommendations for CIDA:
 CIDA needs to help refine the OSS proposal and financing request by asking for clarification from the OSS about its partners’ expectations for the anticipated program results.
 CIDA needs to assign resources to monitor and compare results of all its African programming.
 CIDA needs to support the OSS in seeking Canadian partners, increasing its added value to better position the organization and to promote gender equality.
The evaluation derived one key lesson learned from assessing the program’s performance:
 Financing at the OSS program level should anticipate funding shortfalls and ruptures at the project level so as to retain qualified project personnel and conserve their knowledge, experience, and skills for the project. 28

Evaluation of the
Micronutrient Initiative
Micronutrient Initiative
2008–2014
© CIDA/ANTONIO SUÁREZ WEISE
Evaluation of the M icronutrient nitiative 2008 2014 I -
Why evaluate the Micronutrient Initiative (MI)?
Between 2008 and 2014, CIDA disbursed $150 million to the MI, an international non-governmental organization based in Ottawa. CIDA’s grant funding supports MI programming to helping to eliminate micronutrient malnutrition. MI does this by stimulating and supporting partner countries through strategic partnerships and global leadership in nutrition and micronutrients (MN).
This evaluation examined results between 2008 and 2011 to satisfy a funding requirement for a midterm evaluation. The evaluation also identified lessons learned and provided recommendations for the current grant period and for potential areas of growth for the organization.
 Overall, the MI has made very good progress toward its goals. The MI is a trusted and valued partner at both the global level, where it contributes meaningfully to policy development, and at the country level where it adds value through long-term involvement, innovation, flexibility, and responsiveness to government partners.
 The MI’s contributions to global advocacy are important, particularly for vitamin A supplementation, while at the regional and national levels, its advocacy efforts are linked to policy advances and increased commitments by national governments for the supplementation of vitamin A and zinc, and fortification with iron and folic acid.
 Areas for strengthening include raising awareness globally of the importance of reducing vitamin and mineral deficiencies, increasing interventions for maternal health, and improving the methodology for setting targets in supplementation and fortification programming.
 The MI’s work remains relevant and provides global leadership on the development of guidelines and implementation of micronutrient programming with government and private sector partners.
 To support the MI’s continuing relevance, CIDA could introduce flexibility into the program’s funding mechanisms, allowing the organization to respond to changing nutritional needs and innovations.
 The MI is effectively managed, as shown by its capacity to develop financial and human resources and by its results planning and tracking systems.
 A regional approach optimizes cost-effective use of resources, which in turn helps to produce positive results and innovation.
 A multi-level approach to advocacy is an effective means of leveraging policy and programming commitments by national governments.
 The MI’s long and successful experience in salt iodization and large-scale flour fortification underscores the positive role that the private sector can play in extending and sustaining the program’s aims on a large scale.
Next steps
 CIDA is working with the MI during the development of its strategic plan for 2013–2018 to ensure key findings of the evaluation are incorporated.
 The MI is developing a comprehensive strategy and plan of action for advocacy and partnership, with well-defined expected outcomes. 29

International Legislative
Audit Assistance
Program
ILAAP
2007–2012
© CIDA/JOHN ROBINSON
International Legislative Audit Assistance Program Evaluation 2007-2012
Why evaluate ILAAP?
Between 2007 and 2012, CIDA provided $8.26 million dollars to the Canadian Comprehensive Auditing Foundation’s International Legislative Audit Assistance Program (ILAAP). The ILAAP aimed to improve governance and accountability for developing countries by helping to reduce corruption and to ensure the effective use of public funds. Principally, the funds helped build the capacity of supreme audit institutions (SAIs) in partner countries that will oversee public expenditures and contribute to governmental accountability and transparency.
This evaluation assessed the ILAAP’s performance, measured its effectiveness, and identified results achieved midway through the program’s term. The recommendations and lessons learned will identify alternative ways to meet objectives and serve as a guide for subsequent programming cycles.
 Overall, the program responded well to development issues and needs, making it relevant to participant countries. However, the program’s level of relevance varied on a case-by-case basis depending on whether a country has an established legal and administrative framework to support performance audit activities. Political conditions also affected the extent to which parliamentarians and/or public accounts committees adopted the performance-audit process and the reports recommended by the ILAAP.
 The number and quality of value-for-money audits performed by participating SAIs increased.
 The ILAAP successfully achieved short-term and immediate aims, particularly helping to build the capacity of participant countries to conduct more, better quality, performance audits. The follow-up to
ensure the audit reports’ appropriate use remains to be completed and continued support will be necessary for longer-term sustainability.
 The SAIs of partner countries displayed high levels of local ownership, and in many cases, government ministries provided active direction. However, parliamentary account committees have not, as yet, taken ownership of performance auditing.
 The provision of significant post-program training or mentoring will allow participant personnel to build confidence and participant countries to keep building their capacity to conduct performance audits.
 The formation of regional associations of SAIs created the potential for more regional training to help ensure sustainability and reduce program costs by contracting the fellowship program in Canada.
 The twinning component, which provides linkages between a partner country and a provincial audit office, has been effective in providing mentoring and ongoing support to SAIs. 31

Cida lessons from evaluations

More Related Content

What's hot (14)

Viewers also liked (12)

Similar to Cida lessons from evaluations (20)

Cida lessons from evaluations