Guide for the Monitoring and Evaluation of the Transition of Health Programs

Final Draft
March 2015
This publication was produced for review by the United States Agency for International Development.
It was prepared by James Bao, Daniela Rodriguez, Ligia Paina, Sachi Ozawa, and Sara Bennett for the Health Finance and
Governance Project.
GUIDE FOR THE MONITORING AND
EVALUATION OF THE TRANSITION OF
HEALTH PROGRAMS

The Health Finance and Governance Project
USAID’s Health Finance and Governance (HFG) project will help to improve health in developing countries by
expanding people’s access to health care. Led by Abt Associates, the project team will work with partner countries
to increase their domestic resources for health, manage those precious resources more effectively, and make wise
purchasing decisions. As a result, this five-year, $209 million global project will increase the use of both primary
and priority health services, including HIV/AIDS, tuberculosis, malaria, and reproductive health services. Designed
to fundamentally strengthen health systems, HFG will support countries as they navigate the economic transitions
needed to achieve universal health care.
DATE 2013
Cooperative Agreement No: AID-OAA-A-12-00080
Submitted to: Scott Stewart, AOR
Office of Health Systems
Bureau for Global Health
Recommended Citation: Bao, James, Daniela Rodriguez, Ligia Paina, Sachi Ozawa, and Sara Bennett. March
2015. Guide for the Monitoring and Evaluation of the Transition of Health Programs. Bethesda, MD: Health Finance &
Governance Project, Abt Associates Inc.
Abt Associates Inc. | 4550 Montgomery Avenue, Suite 800 North | Bethesda, Maryland 20814
T: 301.347.5000 | F: 301.652.3916 | www.abtassociates.com
Avenir Health | Broad Branch Associates | Development Alternatives Inc. (DAI) |
| Johns Hopkins Bloomberg School of Public Health (JHSPH) | Results for Development Institute (R4D)
| RTI International | Training Resources Group, Inc. (TRG)

GUIDE FOR THE MONITORING AND
EVALUATION OF THE TRANSITION OF
HEALTH PROGRAMS
DISCLAIMER
The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency
for International Development (USAID) or the United States Government.

CONTENTS
Contents .................................................................................................................... i
Acronyms................................................................................................................. iii
Acknowledgments................................................................................................... v
Executive Summary .............................................................................................. vii
1. Why monitor and evaluate transition? ............................................................. 9
1.1 Purpose and audience.......................................................................................................10
1.2 Outline and presentation of the guide.........................................................................10
2. Towards a common understanding of transition .......................................... 11
3. Principles to support effective M&E of transition ......................................... 15
3.1 Set clear end goals.............................................................................................................15
3.2 Early planning......................................................................................................................15
3.3 Stakeholder engagement..................................................................................................15
3.4 Shared responsibility.........................................................................................................15
3.5 Earmark funding and resources for M&E....................................................................15
3.6 Neutrality and independence of M&E..........................................................................16
4. Conceptual framework for M&E of transition ............................................... 17
4.1 Pre-transition......................................................................................................................17
4.2 Transition preparation process .....................................................................................21
4.2.1 Sustaining a supportive policy environment...............................................22
4.2.2 Creating financial sustainability ......................................................................22
4.2.3 Local stakeholder capacity development....................................................23
4.2.4 Communication to all stakeholders..............................................................23
4.2.5 Program alignment ............................................................................................23
4.3 Institutionalization .............................................................................................................24
4.4 Sustained Services and Outcomes ................................................................................24
5. DESIGNING AN M&E APPROACH...............................................................25
5.1 Why M&E of Transition is Different ............................................................................25
5.2 Defining Objectives & Tailoring Design.......................................................................25
5.3 Establishing time frames, resource needs, & organizational arrangements.......26
6. Designing Instruments for M&E of Transition............................................... 28
6.1 Selection of indicators......................................................................................................28
6.2 Quantitative Indicators.....................................................................................................30
6.3 Qualitative questions........................................................................................................41
7. Transition challenges that affect M&E............................................................45
Annex A: Further reading ....................................................................................51
Avahan Transition..............................................................................................................51
GAVI Transition.................................................................................................................51
Family Planning Graduations in Latin America ..........................................................51
Other readings on transition and sustainability.........................................................51
i

Annex B: References.............................................................................................53
List of Tables
Table 1: Domains of Transition........................................................................................................21
Table 2: Transition Activities Examples .........................................................................................21
Table 3: Data sources for monitoring and evaluating transition.............................................29
Table 4: Guiding questions and quantitative indicators for M&E............................................32
Table 5 – Qualitative questions for M&E.......................................................................................42
Table 6: Challenges in M&E of transition and potential solutions ..........................................45
List of Figures
Figure 1: Transition types and implications for monitoring...................................................... vii
Figure 1: Transition types and implications for monitoring......................................................12
Figure 2 – Conceptual framework for transition ........................................................................19
ii

ACRONYMS
USAID United States Agency for International Development
BiH Bosnia-Herzegovina
BMGF Bill and Melinda Gates Foundation
CHW Community Health Workers
FP Family Planning
GAO Government Accountability Office
GAVI Global Alliance for Vaccines and Immunizations
GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria
HC Health Centers
HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome
HMIS Health Monitoring Information Systems
M&E Monitoring and Evaluation
MNCH Maternal, Neonatal and Child Health
NGO Nongovernmental Organization
PEPFAR President’s Emergency Plan for AIDS Relief
STI Sexually-Transmitted Infections
TB Tuberculosis
USG United States Government
WASH Water, Sanitation and Hygiene
iii

ACKNOWLEDGMENTS
The authors would like to acknowledge the support, cooperation and guidance provided by numerous
individuals critical to the development of this document. In particular we would like to thank Laurel
Hatt, Robert Hecht, Michael Rodriguez, and Eric Sarriot for sharing their unique insights on transition in
different contexts and of different programs that helped inform the frameworks adopted here.
Catherine Connor, Laurel Hatt, Robert Hecht, Ben Johns and Scott Stewart all undertook technical
review of this document. Kathryn Stillman helped facilitate the planning of this product, and provided
efficient and timely management of the process. Further, we would like to extend our thanks to, Carlos
Cuellar, Alison Comfort, Marguerite Farrell, Ann Lion, Alvaro Monroy and Lindsay Stewart for providing
insight into their experiences of health program transitions.
v

EXECUTIVE SUMMARY
Why monitor and evaluate transition?
This guide for the Monitoring and Evaluation (M&E) of transition is motivated by the idea that through
M&E, transition processes can be strengthened in a fashion that develops local political will, helps
mobilize local financial and technical resources, and establishes lines of accountability after donor
support is complete so as to promote long-term sustainability. Currently little is known about transition,
we draw upon experience of monitoring and evaluating (M&E) transition to (i) clarify key elements and
dimensions of transition and how they relate to the longer-term goal of program sustainability and (ii)
present possible indicators, relevant to different health programs and transition arrangements that can
help track transition and offer suggestions on how to select appropriate indicators.
Towards a common understanding of transition
Transition can be defined as the transfer of financial, leadership and programmatic responsibilities for a
health program from a donor to a recipient, according to a pre-defined plan. The terms “transition” and
“graduation” are often used interchangeably, though “graduation” typically refers to a transition
triggered by the recipient achieving a certain level of performance, defined by economic or health
indicators.
Health program transitions vary substantially. We use three different transition experiences (Figure 1) to
demonstrate variations in the type of transition undertaken, and the corresponding need for M&E.
Figure 1: Transition types and implications for monitoring
Transition types and implications for monitoring
Transition type
Types of transition A - GAVI B – Family
planning
C - Avahan
Funding transition X X X
TA & program management transition X X
Service transition X
  INCREASING COMPLEXITY  
Key dimensions for M&E
Sustaining a supportive policy environment X X X
Creating financial sustainability X X X
Local capacity development X X
Communication among all stakeholders X X
Program alignment X
vii

Further we conceptualize the transition process as a series of sequential steps encompassing:
(i) Pre-transition during which donors and potential program recipients engage in discussions
to develop consensus around the program’s transition goals and plans to meet them;
(ii) Transition preparation process that spans four domains (leadership, financing,
programming and service delivery) and five activity areas (namely sustaining a supportive policy
environment, creating financial sustainability, developing local capacity, communicating among
stakeholders and program alignment). As indicated in Figure 1 not all of these activities may be
necessary, depending on the nature of transition;
(iii) Institutionalization which occurs as key features of the program are integrated into the
organizational procedures and behaviors of the recipient organization/s;
(iv) Sustained service and outcomes
Principles and approaches for M&E of transition
Based on experience, a number of principles for the M&E of transition are proposed including the need
to set and agree on clear end goals for the transition, so that they can be monitored, the importance of
early planning and engaging stakeholders in this process, as well as ensuring shared responsibility of M&E
of transition between both the donor and the receiving organization. In addition, resources for M&E
need to be earmarked and an evaluation team should be identified that demonstrates neutrality and
independence.
The guide encourages those commissioning M&E of transition to be clear on why they are undertaking
M&E: (i) to strengthen the implementation of transition strategies, (ii) to assess whether or not elements
of the program are ready for transition, or (iii) to understand the effects of transition over the longer
term? Different motivations for M&E of transition will likely lead to different M&E designs. Given the
lack of rigorous transition evaluations, further investment in transition M&E is needed. Transition
evaluations need to be particularly alert to unexpected consequences of transition. Regular engagement
with stakeholders, and combining open, qualitative evaluation tools with more closed, quantitative ones,
may help identify such unexpected effects in a timely way.
Selection of indicators to track transition should be driven by three criteria, namely the importance of
the dimension of transition being measured, the scientific soundness of the measure, and the feasibility
of collecting data on the measure. Organizational reports, routine data, and primary data collected
through special surveys are all likely to be important in transition M&E. This guide draws upon existing
literature and the conceptual framework developed here, to provide an overview of the kind of
questions that might be addressed as part of transition M&E and suggests possible indicators for each of
these questions. As already noted, given the likelihood of unexpected consequences of transition, M&E
is likely to be strengthened through combining some qualitative assessments together with the
quantitative.
Challenges to M&E of transition
Conducting M&E of transition processes is unlike much other M&E. It is not well established as a form of
evaluation and it seeks to examine a highly complex process, with organizational change taking place at
multiple levels and across numerous stakeholders. M&E of transition is likely to face resistance, both
because people on the ground are busy with implementing the transition process, and also because they
may have a stake in maintaining the status quo. Large-scale transitions can be chaotic and compromise
the quality of the data. There are no perfect solutions to addressing these challenges, but the guide
concludes with some suggestions for managing them.
viii

1. WHY MONITOR AND EVALUATE TRANSITION?
The donor community’s interest in long-term sustainability and the fate of programs once donor funding
is reduced partially or completely has been present since the early 1990s (1) but has escalated recently.
The US Government (USG) has been facilitating various forms of transition for their international
development programs since such programming started. For example, in the early 2000’s, USAID began
the process of “graduating” some countries in the Latin America and Caribbean region from Family
Planning (FP) assistance - in the context of decreasing USG funds for family planning worldwide and
significant drops in total fertility rate in the region (2). The current importance of transition is framed
by an increased focus on sustainability and country capacity to lead and manage programs as program
priorities change, and external funding either declines or ends completely. For example, the PEPFAR
reauthorization in 2008 reinforced the notion that transition must be handled carefully, with the
institution of Partnership Frameworks that aimed to ensure that PEPFAR programs were sustainable
through a renewed focus on “country capacity, ownership and leadership” (3,4). Such focus aligned with
the current context of USAID Forward, whereby USAID looks to build local capacity in order to
channel more development assistance through increased direct support to local institutions. USG
agencies are not the only ones to engage in such discussions. The Bill and Melinda Gates Foundation
(BMGF), The Global Fund to Fight AIDS, TB and Malaria (GFATM) and the Global Alliance for Vaccines
and Immunizations (GAVI) are among several others with growing interest in transition from donor
assistance, as well as long-term sustainability.
Transition and/or graduation from donor support are often portrayed as a daunting prospect, filled with
uncertainty, and threatening the outcomes attained in a particular program area with donor funding. For
example, a recent New York Times article highlighted significant concerns in South Africa surrounding
the anticipated 100 million USD reduction in funding for HIV/AIDS services (5), as PEPFAR refocused its
support to poorer, higher prevalence countries. However transition need not be a negative experience.
Examining experience with the long-term sustainability of USAID-funded health programs in Latin
America and Sub-Saharan Africa, Bossert concluded that “projects for which budgetary sources (i.e. in
the national budget) provides progressively significant contributions to project activities during the life of
the project tended to be more likely to be sustained” (1). In other words, projects for which transition
is planned and managed have greater chances to benefit in the long-term. The transition experience of
the Avahan Indian HIV/AIDS initiative also points to the potential for positive change (6).
This guide for the Monitoring and Evaluation (M&E) of transition is motivated by the idea that through
M&E, transition processes can be strengthened in a fashion that develops local political will, helps
mobilize local financial and technical resources, and establishes lines of accountability after donor
support is complete so as to promote long-term sustainability.
Beyond the few experiences mentioned above, little is currently known about transition - how the
process of transition happens and how to monitor and evaluate whether a transition led to long-term
sustainability or to the partial or complete demise of program outcomes. The frameworks to define and
measure such processes are few, but include efforts to evaluate the sustainability of GAVI-funded
immunization programs in BiH after the country became ineligible to receive funds (7,8) and efforts to
prospectively monitor and evaluate the transition of the BMGF-funded Avahan’s HIV prevention work in
India to local ownership (9). Country ownership and sustainability issues have been discussed in the
context of CDC’s HIV/AIDS programming (10), and GAVI also sought to assess the readiness of
graduating countries to assume responsibility for sustainable financing of immunization processes(11).
Attention has also been focused on the transitioning of health care worker support from PEPFAR to
9

local funding (12). To-date, with the one exception (namely the Avahan transition evaluation), none of
the existing M&E activities have examined transition prospectively or throughout the entire transition
process (9) and there have been no systematic efforts to develop an approach for the M&E of transition.
M&E of transition is unlikely to look like the typical M&E conducted for health programs. Indicators of
service coverage or health outcomes are certainly relevant to understanding transition processes, but so
too, are less commonly examined dimensions such as supportive policy environments, program
alignment, and effective communication between stakeholders.
M&E of transition, while challenging, can offer important benefits. First, the discipline of thinking through
what transition entails and how best to measure it, can provide greater conceptual clarity to the whole
transition process. Second, the M&E of transition can help inform countries undergoing transition about
how best to manage it – both in terms of learning from countries which have undergone the process,
learning from their own transitions over time, and presenting opportunities for course correction. M&E
of transition can help identify potential problem areas before they manifest into more serious issues.
Third, M&E of transition can provide an element of accountability for donors, allowing them to be
assured that the transition process was executed with attention to detail and with overall sustainability
of the program in mind.
1.1 Purpose and audience
Effective monitoring and evaluation of any program or policy depends upon there being a clear
understanding of the program theory—that is the mechanisms through which the intended effects of
the program occur. Accordingly, this guide first seeks to clarify key elements and dimensions of
transition and how they relate to the longer-term goal of program sustainability. Second, the guide
presents possible indicators, relevant to different health programs and transition arrangements that can
help track different aspects of transition and offers suggestions on how to select appropriate indicators.
In the context of pressures to use international health funding more strategically, the issue of transition
is relevant for all international health funding agencies as well as program recipients. This document is
intended to be used by both international funders and program recipients to monitor and evaluate
transition processes.
1.2 Outline and presentation of the guide
Section 2 seeks to develop a common understanding of transition, with respect to terminology and the
dimensions of transition that implicate the design and approach of monitoring and evaluating transition.
To help guide readers throughout the document, three illustrative cases are introduced here and
described according to different dimensions of transition. Section 3 describes a set of principles for the
M&E of transition, which may also and more broadly support a successful transition. Section 4
introduces a conceptual framework for the M&E of transition that sets out various processes and
dimensions of transition, thereby also identifying the types of constructs that one might think about
measuring as part of a transition M&E strategy. This conceptual framework is intended to be sufficiently
flexible to accommodate different types of health programs (HIV/AIDS, vaccines, maternal health),
different types of transition (below in Figure 1), and differing country contexts. Then, Section 5
addresses some key M&E design issues, in particular taking the perspective of someone commissioning
M&E. Section 6 sets out illustrative guiding questions and indicators for the M&E of transition – a
resource upon which readers can draw and adapt based on the particular transition context they are
faced with. Section 7 concludes this guide with reflections on anticipating and responding to challenges
during the M&E of transition. A bibliography of relevant documents that discuss M&E of transition,
sustainability, and country ownership can be found at the end of this document.
10

2. TOWARDS A COMMON UNDERSTANDING
OF TRANSITION
In public health, transition can be defined as the transfer of financial, leadership and programmatic
responsibilities for a health program from a donor to a recipient, according to a pre-defined plan. The
terms “transition” and “graduation” are often used interchangeably. “Transition” can happen for a
variety of reasons – contextual factors such as shift in program or political priorities, change in
leadership; performance factors; or a combination. “Graduation” is often used to refer to a transition
that is triggered by the recipient achieving a certain level of performance, defined by economic or health
indicators. However, performance benchmarks are difficult to define and are often driven by political
and economic pressures, so while some organizations have formal graduation policies, in practice the
distinction between transition and graduation may not be as clear as at first glance.
Planning for and achieving increased “country ownership” or “long-term sustainability” implies that a
transition will take place, but these terms are not specific about the time frame or nature of transition
that may occur. In this report, we use the term “transition” as an umbrella to capture all of the
previously mentioned terms. Furthermore, we recognize that all transitions arise from a mix of political,
economic, or health sector performance factors.
Transitions of health programs can vary according to numerous dimensions. For example transitions
may vary according to:
• The trigger for transition – e.g. graduation based on the attainment of health or economic
performance targets, or donor shifts in funding and programmatic priorities;
• Scale of the program – e.g. geographical scope (regional/national/sub-national), magnitude of
investments or the number of people served;
• Scope of the transition – e.g. transition will differ depending on the type of entity being
transitioned, for example an institution in transition or program in transition;
• Transition timing– e.g. rapid change with no adjustment period versus gradual transitions with the
step-wise transfer of responsibilities.
These factors may be important in terms of determining the feasibility of alternative M&E approaches,
the time frame for M&E, actor dynamics around transition, and the significance of the transition and thus
how much of an investment should be made in transition monitoring.
However in terms of the focus of M&E, it will be important to understand exactly what is being
transitioned. Figure 1 distinguishes between three different transition types. Though these will not cover
all possible transitions scenarios, they represent different types of common transitions that will be
referred to within this document. Type A, illustrated by GAVI program graduations, is exemplified by
the transition of funding arrangements. In these cases program implementation has historically been
carried out largely through government structures, and thus the main concerns focus on whether there
is sufficient funding and stakeholder commitment to maintain support post-transition. For Type B
transitions, exemplified by the transition of USAID family planning and reproductive health programs in
Latin America, funding transitions to local sources, but so too must responsibilities for program
planning, management and technical support. In such cases, in addition to funding diversification and
stakeholder commitment, stakeholder capacity and communication of plans will be important. Finally,
11

Type C transitions entail all of the transition dimensions mentioned above, plus the challenges of shifting
a previously stand-alone program into government and/or local partner systems.
Figure 1: Transition types and implications for monitoring
Transition types and implications for monitoring
Transition type
Types of transition A - GAVI B – Family
planning
C - Avahan
Funding transition X X X
TA & program management transition X X
Service transition X
  INCREASING COMPLEXITY  
Key dimensions for M&E
Sustaining a supportive policy environment X X X
Creating financial sustainability X X X
Local capacity development X X
Communication among all stakeholders X X
Program alignment X
The type of transition will affect which dimensions of transition it is important to monitor. For example,
for relatively straightforward funding transitions, the likely focus of M&E will be on sustaining a
supportive policy environment and creating financial sustainability (as indicated in the bottom half of
Figure 1). By contrast Type C transitions will likely need to pay attention to all of the M&E dimensions
listed.
The boxes below provide a fuller description of the three illustrative cases of transition, describing
multiple dimensions of the process. While these three scenarios capture the principle forms of
transition seen to-date, they are unlikely to adequately reflect all the nuances of transition that exist.
Box 1 – Type A Transition – GAVI
Ex. 1 GAVI support to Bosnia and Herzegovina (BiH) (7)
Donor GAVI funding was used for the introduction of new vaccines and also for injection
safety from 2002 to 2011.
Scale of program The immunization program was implemented nation-wide, in all entities
Financing
arrangements
BiH became ineligible because it passed the GNI per capita threshold. At that time
(2011), there was no policy for a phased or planned transition/graduation.
Change in level of
integration
While services were delivered through public service outlets during GAVI funded
years, the procurement of vaccines was done through UNICEF. Service delivery and
procurement, as well as related functions are all fully integrated within the systems in
each BiH entity.
12

Transition timing and
extent
GAVI funding ended as soon as BiH became ineligible. BiH then took on related
responsibilities.
Program planning,
management and
technical assistance
Program planning, management and technical assistance were largely conducted
within the host country’s own health systems.
Transition trigger Economic growth increased the GNI per capita beyond GAVI’s eligibility threshold,
and at the time this happened, GAVI did not have a graduation policy, and therefore,
there was no official transition. The increase in ineligible countries led GAVI to
develop a graduation policy, which currently includes specification for transition.
Box 2 – Type B Transition – Family planning in Latin America
Ex. 2 Family planning in Latin America – After supporting family planning with financial and technical
assistance across Latin America since the 1960s, USAID recognized the need to be more strategic in allocating
FP funding and more systematically “graduate” countries from assistance.
Donor USAID provided financial and technical support for family planning programs across
Latin America from the 1960s.
Scale of program Family planning programming is widely distributed in all participating countries.
Financing
arrangements
Funding was shifted to a mix of domestic, public and private funding, including
inclusion of family planning in social security, and leveraging the commercial sector
and social marketing to replenish funding.
Change in level of
integration
Family planning programs were implemented through local NGOs in a decentralized
manner. This system remained, and was integrated on a financing element in
legislature and funded by host government.
extent
Transition typically spanned anywhere from 2-5 years depending on the country’s
performance on key indicators, and other political and contextual factors.
Program planning,
management and
Family planning programs were implemented in a parallel system through NGO in a
decentralized manner. Technical assistance was provided to support the full range of
family planning needs including establishment of facilities, training, management,
procurement, education programs, advocacy etc.
Transition trigger Graduation processes were triggered based on total fertility rate and modern
contraceptive prevalence with other criteria dependent on the context. USAID
country missions worked with transition teams from Washington, DC to create a
transition plan that typically spanned 2-5 years (2).
Box 3 – Type C Transition – Avahan HIV prevention in India
Ex. 3 AVAHAN in India – In 2005 the BMGF committed 350 million USD to address the spread of HIV/AIDS in
India through the Avahan program, focusing on prevention strategies with at-risk populations. The program was
implemented through cascading contracts with international and local NGOs.
Donor BMGF supported all aspects of program implementation.
Scale of program Approximately 200 targeted intervention programs were implemented across six
states with the highest HIV/AIDS prevalence, as well as programs for condom social
marketing, support to HIV prevention among truckers, and mass media
communications for behavior change.
13

Financing
arrangements
The program was initially fully funded by BMGF. Gradually, during the transition
period, the Indian government fully absorbed program funding.
Change in level of
integration
Avahan had been implemented through a separate and parallel delivery system
involving BMGF staff, international and local NGOs. This system was transitioned
fully into the government system.
extent
From initiation of transition planning to completion of transition took 7 years.
Programs were gradually transitioned over 4 years.
Program planning,
management and
Avahan was implemented through a parallel delivery system supported by staff from
the BMGF, and local and international NGOs. Transition into the government system
passed these responsibilities onto the program recipient.
Transition trigger The second phase of Avahan focused on transitioning the program to it’s ‘natural’
owners, primarily the Government of India.
14

3. PRINCIPLES TO SUPPORT EFFECTIVE
M&E OF TRANSITION
The team has drawn on its collective experience with transition to propose six principles for guiding the
M&E of transition. While these principles are focused on ensuring effective M&E of transition directly
they may also contribute the successful planning and implementation of transition.
3.1 Set clear end goals
Clear end goals are critical to guiding plans for M&E. Active and ongoing engagement with stakeholders,
especially the donor and program recipient organizations, is important to clarify how everyone envisions
the future of the program and its role in sustaining any gains in health. To the extent possible, end goals
should be mutually agreed among stakeholders and may be set in terms of the continuation of all or
some of the services previously supported by the donor. It may be planned for service coverage not
only to be sustained but to grow (particularly if there is an established upward trend in coverage).
Alternatively goals may be specified in terms of health outcomes, but explicitly acknowledging the
potential for change in services provided. Sometimes non-health outcomes (eg. income generating
opportunities, or female education) may also be important goals.
3.2 Early planning
Effective M&E of transition requires careful and thoughtful planning. Early planning allows for regular and
consistent monitoring of the transition process as well as evaluating the transition preparation activities
being conducted, including gathering baseline assessments to determine impacts after transition.
3.3 Stakeholder engagement
As part of transition M&E, it is critical to engage all stakeholders and secure commitments from key
actors to the process and the use of evidence emerging from M&E. It is especially critical for the
program recipient receiving the transitioned program to have a vested interest in the M&E process
partly because it is necessary to ensure access to key data sources post-transition but also because of
the central role this organization will play in acting upon M&E evidence.
3.4 Shared responsibility
Related to stakeholder engagement, it is important for donor and program recipients to identify and
allocate roles and responsibilities for monitoring and evaluating a successful transition in light of mutually
agreed end goals.
3.5 Earmark funding and resources for M&E
Programmatic transition is separate and distinct from programmatic service delivery and as such needs
specific earmarked funding, so too does the M&E of transition. Successful M&E of transition requires
15

budget allocations within the overall transition budget commensurate with the goals of the transition
plan.
3.6 Neutrality and independence of M&E
A critical component for a successful M&E plan is to guarantee the neutrality of the results and the
independence of the evaluators so that all stakeholders view results as unbiased. This can be achieved by
engaging external evaluators, but evaluation teams composed of donor and program recipient
representatives can also fulfill this role. Nevertheless, the evaluation team must function independently
and in service of the overall transition and M&E goals, without outside influence from vested
stakeholders.
16

4. CONCEPTUAL FRAMEWORK FOR M&E OF
TRANSITION
We sought to base this guide for M&E of transition upon a conceptual framework that appropriately and
broadly detailed the transition process for health programs. Given the lack of such frameworks in the
existing literature, we sought to develop a framework based upon our experience and the available
literature. The proposed framework seeks to model the transition process in a comprehensive fashion.
It details a number of tasks and roles that will likely need to be completed during transition, while
recognizing that different actors will take on these tasks in different circumstances. The M&E team can
use this conceptual framework to identify those elements of the transition process which are relevant to
the context in which they are working, and thus to determine which aspects of transition they will seek
to monitor.
Transition can be thought of as a process, consisting of pre-transition activities that establish the critical
elements for transition, and an on-going iterative process of transition preparation and transition
activities, which shift program responsibility from the donor to program recipient (see Figure 2).
During the transition process, activities within five domains (explored below) help create the conditions
to anchor the necessary responsibilities, rules, norms and structures into the program recipient
environment to deliver program services at a level as defined by the transition goals – we refer to this
process as institutionalization. This process enables sustained delivery of program activities and sustained
health outcomes. The ultimate goal is for local stakeholders to take responsibility for the program
(programming, leadership, financing and service delivery) with a reduced need for external support.
4.1 Pre-transition
The first stage is pre-transition, where activities set the stage for transition. Either donor or in-
country program recipient, depending on the situation, may trigger the transition process, which will
influence the nature of transition. For example, a forced transition exit casts different dynamics in pre-
transition planning in comparison to a mutually agreed upon and planned transition process. Donors
may signal the need for a transition on the basis of target indicators being met or political reasons. In
other instances, program recipients could initiate transition planning, having the foresight to try and
develop a sustainable program. During this process, and under ideal circumstances, donors and potential
program recipients (government agencies, non-governmental organizations, community partners, service
providers, etc…) engage in discussions to develop consensus around the program’s transition goals and
plans to meet them. With transition planning in the early stages of program development, sustainability
and transition can be kept in mind as the program is developed (13). Through an open and transparent
approach, this process determines the stakeholders to be involved in transition, reviews the reasons
that transition is occurring (e.g., loss of funding, planned graduation based on indicators), uses consensus
among stakeholders to establish how transition and sustainability will be achieved, allocates an
appropriate budget for transition activities and creates an agreed-upon transition plan. These pre-
transition activities are influenced by the nature of transition. Ultimately, strong pre-transition planning
may help build commitment and overall support for the transition process.
17

Figure 2 – Conceptual framework for transition
19

4.2 Transition preparation process
The pre-transition process creates a foundation for the transition process itself, when transition
plans are set into motion through various activities. The activities broadly span four areas (Table 2) and
help drive the transition of domains: leadership, financing, programming, and service delivery
(Table 1). These activities constitute the transition process itself and help create the conditions to
transition these domains from donor to program recipient. Accordingly, these activities facilitate the
process of anchoring the norms, regulations, structures and processes necessary to sustain service
delivery with reduced dependence.
Table 1: Domains of Transition
Table 1: Domains of transition
Domain Description and examples
Leadership High-level leaders must accept that the transition process is actually occurring, and
communication and other activities (see Table 2) should occur to ensure that leaders are
aware of the need for transition, and do their necessary parts to prepare for and support
transition.
Financing Activities relating to creating financial sustainability during the transition period so that the
program recipient is financially sustainable as past sources of funding are eliminated.
Funding often comes from multiple sources and as such, activities may include lobbying to
secure funding from new sources and, creating and altering financial mechanisms for
improved sustainability.
Programming Responsibilities for program management, such as day-to-day operations, as well as staff
management, funding, reporting requirements, monitoring and evaluation and other
administration must be transitioned, to the extent that such functions were previously
provided by donors. Capacity assessments can help diagnose competencies and signal the
amount of capacity building and training required to transition programming.
Service
delivery
In instances where donors, and not local organizations, have been directly responsible for
service delivery, the local program recipient may have to take responsibility for the logistics
of service delivery, including human resources, commodity procurement, community
outreach and other elements related to the program services itself.
Table 2: Transition Activities Examples
Table 2: Transition activities examples
Dimension Activities
Sustaining a
supportive policy
environment
Building stakeholder transition commitment, strategic communication, community
outreach, lobbying for changes in national policy (e.g., procurement laws), tracking
recipient commitments to policy (eg. through funding allocations)
21

Creating financial
sustainability
Financial analysis, cost-effectiveness analyses, lobbying for earmarked funds,
investigating alternative funding streams, applying for grants, promoting private
sector growth, expanding health markets, developing multi-year financial plans.
Local capacity
development
Professional training, peer assessments, organizational assessments, process
consulting, development of job tools and best practice guides, performance
contracting and a variety of partnership types
Communication
among all
stakeholders
Regular communication to key stakeholders through face-to-face communications,
written communications to broad array of stakeholders, progress meetings,
newsletters and list-serves etc
Program alignment Alignment of processes for procuring and distributing commodities and equipment,
modification of financial mechanisms (cost structures, disbursement processes,
financial reporting requirements), shifts in clinical guidelines, establishment of
common supervisory structures, and adoption of health worker norms (e.g., team
composition, educational requirements for different roles).
The five activity areas are:
4.2.1 Sustaining a supportive policy environment
Ideally transition plans are executed in an environment where existing policies and involved stakeholders
are committed and supportive of achieving the overall objective of the transition and health program.
Transition is often met with resistance so it is crucial to conduct activities strategically to build political
commitment and support. In instances where leaders can be held accountable to their constituents,
creating wide public support for a health program may encourage leaders to visibly support the
sustainability and transition of a health program. In other instances, “soft” approaches such as strategic
communication of benefits can be done to reason with influential stakeholders. Signals of a supportive
policy environment may include the post-transition program being embedded in national policy, or
specific program goals being reflected in national and/or sub-national plans. Sometimes existing policies
may undermine program sustainability, for example, existing policies inhibit effective procurement
processes, which would be a target for change.
4.2.2 Creating financial sustainability
Existence of secure and diversified funding is central to the sustainability of a health program (14,15).
The burden of securing this funding is dependent on the context, but responsibility may fall in part to
the program recipients themselves (e.g. for FP in Latin America, some NGO services were not eligible
for inclusion on social security, and thus relied on the commercial sector to replenish funding) and the
donor. As there can be multiple donors and funding sources for a program, coordination is central in
organizing how funds are raised and shared. An understanding of a country and program’s current and
future needs (e.g., using a resource plan) can better conceptualize the funding situation to key audiences
(16). Ultimately, funding is related to contextual issues (economic conditions, political will, competing
government priorities, capacity) and is affected by the donor landscape, where presence of generous
donors may discourage acceptance of financial responsibility (17).
22

4.2.3 Local stakeholder capacity development
Shifting health program responsibility from donor to program recipient means that departing capacity
previously supplied by donors must be replaced with local capacity. Organizational capacity assessments
may diagnose existing competencies and areas in need of investment to reach sufficient capacity for
sustaining the health program (18). When capacity is insufficient, then capacity-building activities should
be initiated to develop the necessary components to the extent that it’s sufficient to continually deliver
program activities (18). Capacity goes beyond the idea of having the necessary skills and tools to deliver
program activities, but also comprises staff, facilities, structures, and systems. A diverse range of
activities can be undertaken, depending on the needs and resources present (see Table 2)(19).
4.2.4 Communication to all stakeholders
Transition is a process that inherently involves shifts in power and authority, as a result, transitions
often face considerable stakeholder resistance. Timely, transparent, and appropriately disseminated
communications play a key role in persuading stakeholders and forming a group of proponents to
support transition. A communication strategy can help identify key audiences and stakeholders
(policymakers, NGOs, service providers, media, local leaders, program beneficiaries) to be engaged
throughout transition (20). Transparent, understandable and strong communication of the transition
plan helps align expectations, communicate common goals and can facilitate building positive
relationships among these key audiences. These activities can help overcome common risks of transition
such as being overwhelmed by tension and noise from confusion and misinformed audiences, and
ultimately help overcome resistance to transition (20). Also, communication of the transition plan needs
to occur at multiple levels, from the donor to senior management within the program recipient, as well
as from program recipient to front line workers. Poor communication with front-line workers can
create resistance due to shifting priorities or changing values, for example a stronger emphasis on cost
recovery by frontline workers may generate resistance against transition.
4.2.5 Program alignment
Programs may need to undergo a process of harmonization with existing services as they transition, for
example this can include the adaptation of program services, but also the implementation of common
arrangements for planning, management, financial reporting, and M&E, so as to integrate with the
national program or host environment. Programs activities can be adapted, completely removed, or
remain unchanged in accordance with the program recipient context (e.g., demand for specific programs
may not be demonstrated at the local level, or capacity may be insufficient) (21). The implementation of
such alignment processes can constitute a significant task. Examples are provided in Table 2.
The five activity areas above are closely interconnected and reflect complex adaptive relationships that
influence one another. For example, communicating transition plans to stakeholders is closely related to
creating and sustaining a supportive policy environment, while the policy environment implicates a
program’s ability to secure funding and align programs with existing ones.
These groups of activities drive the transition of leadership, financing, programming and service
delivery forward so the program recipients can eventually take full responsibility over these domains,
and the health program as a whole. Assessments of each domain during the transition process can help
determine areas in need of improvement among program recipients. Depending on assessments results,
additional activities can be implemented to push transition forward to achieve the desired transition
outcomes.
Transition processes often occurs iteratively, and M&E findings during transition can spur reflection,
learning and adaptation to create an improved and more sustainable program (20). Whether this
23

learning and adaptation occurs largely depends on the scale and timing of transition among different
program sites, as well as the extent and nature of M&E. Programs with shorter timelines may allocate
less time for transition and if multiple sites exist, they are likely to transition simultaneously. Thereby,
shorter transitions reduce a program’s potential to use prior experiences from the same program to
inform other transition efforts.
4.3 Institutionalization
As transition activities occur, key features of the program are integrated into the organizational
procedures and behaviors of the recipient organization in a process that we refer to in this document as
institutionalization. Once program activities are institutionalized they become part of the routine
operating procedures of the recipient organization, and are fully reflected in budgets, norms and
guidelines. In this anchoring process, key program elements may be retained with their original form,
staying faithful to the design of the program, but to some degree, program elements are likely to evolve
or adapt to the new system environment, for the intended benefit of integration and sustainability.
4.4 Sustained Services and Outcomes
Typically, transitions aim to sustain or improve health outcomes during transition. However, some may
expect that in a transition, the quantity and/or quality of services delivered may suffer, depending on the
type of program being transitioned. If the activities/interventions are effective then resultant health
outcomes have a higher likelihood of being sustained post-transition. Nevertheless, the original program
services do not need to be necessarily sustained in order for sustained outcomes. The adaptation of
program activities may not sustain activities, but innovation in technologies or program organization
could result in sustained (or improved) outcomes while original services themselves are not sustained.
24

5. DESIGNING AN M&E APPROACH
There are many good guides to health program M&E (see for example Gage et al. 2005 (22), Finn
2007(23), and UNAIDS 2000(24)). These guides set out the fundamentals principles of M&E and provide
an overview of available indicators for tracking health programs. While these resources can inform the
design of a solid M&E approach for transition, there are a number of reasons why M&E of transition
presents different types of challenges. This section explains these differences and reflects on their
implications. We do not provide a full description of the basics of M&E but refer readers to these other
sources, and instead focus our attention on the kind of information that those commissioning M&E of
transition might benefit from.
5.1 Why M&E of Transition is Different
While the ultimate goal of transition, namely the sustaining or perhaps even enhancement of services
and outcomes, reflects constructs that health program evaluators are accustomed to measuring, many of
the other dimensions of transition will present challenges for most health program evaluators. Routine
program monitoring may provide a source of information for transition M&E but it is unlikely to be
sufficient. M&E of transition requires measures of factors such as program alignment, the presence of a
supportive policy environment, organizational capacity and effective communication. Such dimensions
are much less commonly assessed in health programs. Further, likely measures of these factors may be
embedded in various data sources that are scattered across different stakeholders and often cannot
easily be identified through routine channels. Indeed M&E of transition may require the development and
implementation of special data collection tools.
Health programs typically emphasize rigorous and scientific M&E methodologies to ensure high data
quality, and accuracy. However, transition processes may differ in the sense that they rely heavily on the
effective management of relationships between different stakeholders, most notably between donors
and the recipient organization. Thus, when designing an approach for M&E, it may be beneficial and
necessary to allow for a trade-off; between scientifically rigorous data collection and quality metrics on
the one hand, and involvement of the right stakeholders. Poor data collection processes or quality can
gradually be improved, but there is no substitute for involving the right stakeholders during transition.
Each transition is unique. The ideas presented here provide general guidance for stakeholders planning
transition M&E but adaptations to specific contexts – nature of program, size of program, time frame
etc. – should be anticipated.
5.2 Defining Objectives & Tailoring Design
Transitions may be monitored and evaluated for different reasons, for example stakeholders may wish
to:
• Improve implementation of transition activities such as development of financial sustainability
strategies, program alignment, communication;
• Ascertain whether or not different elements of the program are ready for transition
• Learn from what worked well, and what worked less well, and draw implications for policy and
planning
• Understand the effects of transition on a program over the longer term.
25

Monitoring and evaluation approaches will be most successful when stakeholders have discussed why
they are conducting M&E and how information coming from M&E will be used. Sharing of M&E findings
during the transition process, if done well, can help facilitate communication between all concerned.
Monitoring is typically thought of as an ongoing process to get timely feedback on progress towards
final goals and objectives. Thus monitoring may be suitable to get real time information about the
implementation of transition activities (sometimes referred to as process monitoring), or readiness of
program elements to transition. Such information can help inform transition strategy. For example based
on such monitoring findings, stakeholders may decide to slow down or speed up the transition process,
or focus preparatory efforts on particular aspects of transition, which seem to be going less well.
Typically monitoring approaches will not seek to explain why different results are being observed.
Evaluations are typically more rigorous processes, often implemented by an independent, neutral party.
They may inform transition strategy but can also be used for accountability purposes (Was the transition
managed effectively?) and for learning purposes (Which transition preparation activities were most
important in leading to a successful transition?). Accordingly evaluations often seek to answer questions
not just about what happened but also why it happened.
Rigorous and in-depth evaluations of transitions are sorely needed given the very limited evidence base
concerning which strategies support successful transition. Such evaluations can inform decision-making
not only in the country where the transition is taking place, but findings may also be transferable to
other contexts. An evaluation can also commission more in-depth studies that help to explain the trends
observed in the monitoring indicators. For example, perhaps one domain of transition readiness appears
to lag behind the others, qualitative research with program managers or health workers could perhaps
explain why this is the case. Supplementary qualitative research may be particularly important to
understand emerging behaviors as a consequence of transition: for example perhaps people’s demand
for immunization services increase as they no longer perceive vaccination services to be associated with
a foreign government, or services are declining due to demotivation of health workers.
To the extent that evaluations seek to assess the overall success of transition, it is important to be clear
on what a successful transition will look like. As depicted in the conceptual framework, success might be
conceived in terms of final distal outcomes (were health outcomes sustained?), more intermediate
outcomes (were services sustained?), or quite proximal measures (were key features of the program
continued post-transition?). While all of these levels of outcomes may be important, it likely to be
prohibitively expensive to measure them all and measuring some of them and not others might not
reveal the whole picture. For example, simple aggregated measures of health outcomes or service
coverage may fail to reveal that there have been substantial shifts in the nature of the program, and
perhaps a very different mix of clients is being served now to previously.
It is very difficult to anticipate all the likely effects of transition in advance, but systems thinking
approaches may help stakeholders consider possible unanticipated consequences and emergent
behaviors (25). Such thinking will be most productive when undertaken collaboratively between the key
stakeholders involved in the transition, notably including the donor and the program recipients (whether
this be government or a local NGO), as well as other affected parties such as health workers. Through
joint discussions it may be possible to identify possible, previously unforeseen, consequences of
transition as well as determine a collective vision of what a successful transition would look like.
5.3 Establishing time frames, resource needs, & organizational
arrangements
The earlier that planning for M&E of transition can begin, the better, and preferably M&E should be
planned at the same time as the transition process (and for that too, longer time frames are preferable).
26

Nonetheless it is inevitable that in some instances rapid decisions regarding transition will be made: M&E
under such circumstances is likely to be more difficult to implement, but is arguably even more
important in terms of understanding readiness for and consequences of transition. There are also
questions concerning how long M&E should continue for after transition. At this point little is known
about likely program trajectories post-transition and hence, appropriate time frames during which to
track programs. The Avahan evaluation included only one year of post-transition activity (9) but the
studies by Bossert (1) considered a three year post-transition time frame, that is likely to provide a
more accurate assessment of long term sustainability.
For most transition M&E, it will be critical to ensure that comparable data are available both before and
after the transition. This may not be a straightforward task, particularly when the transition involves
integrating a donor-funded project into a government health system, as this will frequently involve
shifting to new routine information systems that may capture data differently. In such circumstances it
will be important to assess in advance, the extent to which key indicators will be available post-
transition, and to put necessary agreements in place to ensure access to such data (where available) or
create mechanisms to generate comparable data (as necessary).
Resource needs for M&E of transition will inevitably vary according to the scale of the program, the time
frame of transition, and the objectives of M&E. Longer term evaluations of large scale transitions may
cost US$2-3 million, however more focused assessments that are able to make use of secondary data for
monitoring, perhaps complementing this with more in-depth investigation of specific issues could be
conducted within a significantly more modest budget. Ideally resources to support M&E should come
both from the donor and recipient organization, as a means to promote joint engagement in and
commitment to the M&E process.
In order to maximize the potential that M&E data are used to inform decision making, it will be
important to link M&E findings to existing decision making structures. For example if there is a joint
oversight committee responsible for planning and implementing the transition process, then the M&E
team could report to them. If such standing structures do not already exist then it may be worthwhile
creating a special M&E advisory committee that is responsible for guiding the M&E process, but also
ensuring that findings are acted upon. Such advisory committees would benefit from diverse
membership, and the inclusion of multiple different types of stakeholders in the M&E process. Program
transition may face some specific challenges in terms of ensuring the reliability of data. Frequently
program implementers, at least initially, will be resistant to transition, and hence may be motivated to
exaggerate the negative consequence of transition or stress the lack of appropriate preparation for
transition. It may be particularly important therefore to question the reliability of routine administrative
data collected for M&E, and also to ensure appropriate triangulation of qualitative data so that the views
of multiple transition stakeholders are cross-checked against each other.
27

6. DESIGNING INSTRUMENTS FOR M&E OF
TRANSITION
In this section, we present guiding questions and illustrative indicators that can be considered and
adapted when designing the approach for M&E of transition. Both quantitative indicators as well as
qualitative questions have been included below as they are complementary methods necessary to fully
explore transition. Quantitative indicators can identify changes that have occurred due to transition,
show trends over time, and help track whether transition milestones are being met. Qualitative
methods help describe the transition experience, explain why changes have occurred and their
repercussions, what feedback and adaptation are taking place, and whether programmatic characteristics
have been institutionalized, among others.
6.1 Selection of indicators
The selection of indicators for monitoring should be driven primarily by (26):-
• The importance of what is being measured
• The scientific soundness of the measure
• The feasibility of obtaining data on the measure
In terms of importance, ideally transition planning will have developed a clear logic model (or program
theory) describing the anticipated linkages between different transition preparation activities and
outcomes. In such a context it will be rational to tie the selection of indicators to the main constructs
covered in the logic model, ensuring a balanced set of indicators across different aspects of the
transition. For example, using the generic conceptual framework presented in this document,
monitoring indicators could seek to capture a variety of pre-transition activities (such as development
of a transition plan), aspects of the transition preparation process (such as measures of local
stakeholder capacity, program alignment, communication, etc.), the extent of program
institutionalization, and different levels of outcomes and how they are sustained over time. In situations
where there is not a clearly defined transition plan, it will be necessary to seek to determine what
activities are currently underway to prepare for transition and how it is envisaged that they will
contribute to transition.
There are a number of different measures of scientific soundness (see Box 4). In practice, for transition
processes there are likely to be relatively sound measures of services and outcomes, but much greater
uncertainty about how best to construct indicators of transition readiness, program alignment, capacity
development, a supportive policy environment and financial sustainability. Those involved in indicator
selection should also choose indicators based on their understanding of the program’s theory of change
or logic model.
Box 4 – Scientific soundness of measures
28

Validity: Does the measure actually measure what it is intended to measure? Including:
• Face validity – does the measure make sense logically and clinically?
• Construct validity – does the measure correlate well with other measures of the same
construct?
• Content validity – does the measure capture meaningful aspects of the construct being
measured?
Reliability: Does the measure provide stable results across various populations and circumstances?
The measure should produce consistent results when repeated in the same populations and settings,
even when assessed by different people or at different times. Measure variability should result from
changes in the subject of measurement rather than from artifacts of measurement.
Box 5 – Assessing transition readiness in the Avahan program
The transition readiness survey examined a range of indicators that sought to assess how well prepared
the targeted interventions (TI) run by Avahan were for transition. Through a literature review and
conceptual framework (9), three elements of transition readiness were identified: 1) capacity, 2)
alignment, and 3) communication. Capacity indicators captured key operations of the TI, such as linkages
made with government health facilities, formation of community groups and functioning of crisis
response committees. Alignment indicators measured levels of preparations made by the TI towards
meeting NACO norms in areas such as team structure, budgeting and reporting. Communication
captured whether staff were informed about the transition, transition plans incorporated staff inputs and
project coordinators received training for the transition. In addition to interview questions, the survey
also included document review of the Avahan management information system (MIS) from which we
abstracted relevant indicators. For each indicator gathered from interviews and documents, we defined
3 levels (0=low, 1=medium and 2=high) of transition readiness based on how well prepared TIs were for
transition. For example, we classified the extent to which TIs met NACO norms, established linkages
with government health services or had informed staff about the transition. The transition readiness
survey captured 21 indicators, including 18 from interviews and 3 from document reviews.
Feasibility and ease of obtaining data on the measure is of obviously of critical importance. Data may
originate from a number of sources, each with their respective advantages and disadvantages. The data
sources to be used ultimately depend on data accessibility and the type of information required. Multiple
options exist (see Table 3).
Table 3: Data sources for monitoring and evaluating transition
Data sources for monitoring and evaluating transition
Data source Examples Advantages/disadvantages
Administrative
records
Health organization records on
number of people involved in
training, existence of transition
plans
Advantages: Inexpensive and available on regular basis
Disadvantages: Poor reliability and completeness
Routine
information
systems
Budget and expenditure data,
service delivery data
Advantages: Inexpensive and available on regular basis
Disadvantages: Poor reliability and completeness
Large-scale Demographic health surveys on Advantages: Routinely administered surveys are often of
29

surveys service coverage better quality
Disadvantages: May be administered infrequently and
therefore data is often dated or misaligned from program
activity timing
One-off surveys One-time survey on awareness of
health services administered by
government agency
Advantages: Provides highly relevant information that is
unlikely to be available elsewhere
Disadvantages: Expensive to administer
Qualitative
research
Interviews with key stakeholders,
focus group discussions with
service providers
Advantages: Provides insight into why changes have taken
place, and also felt experiences of the transition process.
Can highlight diversity of experiences
Disadvantages: Can be time consuming and expensive to
do well, takes skilled researchers to analyze proficiently
Linked to the question of the feasibility of accessing data for indicators, is the question of the frequency
with which such data should be collected and analyzed. Some of dimensions of program transition can
change very rapidly, for example political commitment to a program might change overnight with the
election of a new government, or a media story that compels high-level political attention. However
other dimensions of transition, such as local stakeholder capacity, is likely to change more slowly. The
other side of this question concerns how frequently it is practical for stakeholders to receive and react
to feedback on the transition process, and the time frame over which the transition is occurring.
6.2 Quantitative Indicators
The following tables present a series of questions and example indicators that can be used to guide the
development of a transition M&E plan. There are four tables each representing one of the periods of
the transition as reflected in the conceptual framework (i.e. pre-transition, transition preparation,
institutionalization, and sustained activities and services) as well as the relevant dimensions of transition.
The guiding questions are intended to apply to many different types of transitions, however we use the
archetypes described in section 2 above to distinguish between dimensions that may be particularly
relevant to different types of transition.
The indicators listed here are drawn from various sources, including specific work conducted in
evaluating transition, other sources of programmatic indicators that we have adapted to apply to
transition, and from our collective experience exploring these issues. Where applicable, we have
included useful references to explore indicators further. While many of the indicators suggested here
Box 4: A learning approach to transition
The GAVI Alliance has adopted a learning and trial-and-error approach in the development of
their Graduation policy (36). As GAVI recipients started to become ineligible for renewing GAVI
funding agreements, GAVI saw the need to develop an explicit policy for graduation, including for a
transition period. Whereas the first countries that became ineligible for GAVI support did not
benefit from a transition period(7,8), GAVI subsequently commissioned evaluation pilots of transition
(11). The lessons that emerged from these pilots shaped how GAVI focused and defined its
graduation policy, therefore creating procedures for smoother hand-over and increasing the chances
for long-term sustainability.
30

are applicable to any type of transition, a few program-specific indicators have been included as
illustrative examples.
There are two important points to note. First, the indicators suggested here do not necessarily apply to
every instance of transition. Second, this is not an exhaustive list of indicators. Evaluators need to be
sensitive to the fact that each transition is likely to have different characteristics and indicators proposed
here would need to be adapted to reflect the relevant context, stakeholders and process. Further, each
transition may generate its own set of additional questions and indicators to be addressed. Some
thought should also be given to determining which indicators should be collected pre- or post-transition,
or if both time points are relevant (e.g. budget allocations, staffing, key outcomes, etc.)
31

Table 4: Guiding questions and quantitative indicators for M&E
Pre-transition
Dimension Guiding question Example Indicators
Indicator
References a
Pre-
Transition
Relevant to all
transition
archetypes
To what extent has a
core set of transition
stakeholders been
identified?
• Donor and program recipient have agreed upon key stakeholders for
transition, incl. beneficiaries, civil society, etc.
• % of key stakeholders that have been participated in planning events
• A transition team representing key stakeholders has been established
• The transition teams meets on a regular basis to discuss transition
preparations
Bennett et al. 2011
(9)
Johnson et al. 2004
(27)
To what extent has this
core set of transition
stakeholders agreed on
transition objectives?
• % of key stakeholders that have participated in transition planning events
• Key transition goals and objectives agreed upon and documented (e.g.,
through a memorandum of understanding)
To what extent have the
transition objectives
been planned for,
including monitoring and
evaluation?
• Transition plan has been agreed upon and documented, including transition
timelines
• Donor and program have agreed on a plan for M&E of transition, incl.
funding
• Transition timelines reflect critical points for reviewing activities and findings
from M&E
• A transition manager at donor organization has been assigned
• A transition manager at program recipient has been assigned
• % of goals from transition timeline that are met on a timely basis
To what extent have
budget allocations been
made for transition,
including monitoring and
evaluation of transition?
• Transition budget has been developed at donor organization and program
recipient
• % of donor organization transition budget that has been funded
• % of program recipient transition budget that has been funded
• % of transition budget is earmarked for M&E of transition and is reflected as
a line item in transition budget plans
32

Transition Preparation Process
Dimension Guiding question Example Indicators a,b
Indicator
References c
Communication
among all
stakeholders
Relevant to
transition
archetypes
B & C
To what extent has the
transition plan been
communicated across the
organization and to all levels?
• % of donor and program recipient staff that are aware that
transition is taking place
• % of donor and program recipient staff that understand the
timeline and necessary steps for transition
• % of community leaders that have been informed of transition
plans
• # of regular meetings for key stakeholders for transition updates
• # of meetings about transition aimed at program beneficiaries held
at donor and/or program recipient
Bennett et al.
2011 (9)
To what extent is there a
common understanding of the
transition plans and goals among
transition stakeholders?
• Donor and program recipient staff agree on and endorse activities
necessary for transition
• Has program level transition planning been developed with staff
inputs?
• Program staff have received training on transition activities
relevant to their role
• Transition team meets on a regular basis to review progress
Bennett et al.
2011 (9)
Sustaining a
supportive
policy
environment
Relevant to all
transition
archetypes
To what extent is there clear
commitment from the political
level for program service
delivery over the long term?
• Program is mentioned in national health plan
• Program has been mentioned in high-level gov’t speech
• Program is integrated into national child health policy (MNCH)
• Constitutional guarantee of access to food (Nutrition)
• Program recipient is a good advocate at-risk populations (HIV)
Bennett et al.
2011 (9)
Johnson et al.
2004 (27)
Fox et al 2014
(28)
33

Indicator
References c
To what extent is there
transparent government
leadership and management?
• Have new guidelines provided any flexibility on budget, based on
realities on the ground?
• Do the new guidelines allow any exceptions to operating norms
(other than budget) based on realities on the ground?
• Has there been a change in oversight of the program due to the
transition?
• Are there clear lines of accountability for meeting program
targets?
Bennett et al.
2011 (9)
Johnson et al.
2004 (28)
To what extent have local
authorities incorporated the
demands of program service
delivery into their routine
operations?
• Local authority leaders meet regularly with program staff to
discuss program status
• Program activities are integrated into local operational plans
• Local authority leaders engage program beneficiaries to identify
ongoing or unmet needs
To what extent do local
stakeholders believe that the
program is a valuable and
effective investment of their time
and resources?
• Local community leaders promote program to community
members and support attendance
• Health staff promote program participation
• Has the relationship between clients and service providers
changed due to the transition?
• Program implementers believe that program recipient has the
same or higher level of commitment toward the program as
compared to donor
Bennett et al.
2011 (9)
34

Indicator
References c
Local capacity
development
Relevant to
transition
archetypes
B & C
technical, managerial and
financial capacity within the
program recipient to effectively
deliver key health program
services?
• % of program staff who participated in development of last annual
plan
• % of program meetings that regularly take place
• % of staff that attend program meetings
• % of program staff who can accurately complete necessary
reporting forms
• % of required supervision sessions that occur
• % of program staff who have participated in capacity development
activities to maintain/improve their technical skills in the last two
years
• Is there a regular staff member who can perform basic accounting
tasks for the program?
• Has supervision of your work changed due to the transition?
Bennett et al.
2011 (9)
To what extent have any
shortages in capacity been
identified?
• Capacity needs assessment has been conducted at the program
recipient
• % of supervisory and managerial position vacant at program
recipient prior to transition
• % of program recipient staff qualified for financial management
• Has there been any change in the team structure?
• Are the training needs of peer educators assessed?
Bennett et al.
2011 (9)
Johnson et al.
2004 (27)
Sarriot et al.
2009 (29)
35

Indicator
References c
To what extent are
training/capacity-building
activities occurring or planned to
address gaps in capacity?
• Capacity assessment results have been reviewed by the transition
team and training activities are planned to address capacity
shortages
• % of training activities completed where capacity shortages were
identified
• % of program staff who have attended training/capacity-building
activities specific to their role
• Do peer educators receive skills and leadership training (beyond
general orientation training)?
Bennett et al.
2011 (9)
Create financial
sustainability
Relevant to all
transition
archetypes
To what extent does the
program recipient have
transparent systems to develop
and maintain budgets and
expenditures?
• % implementers and/or districts with costed plans to absorb
program
• % implementers with an audit of their financial records
• Has the program recipient received a clean financial audit report
in the last X months?
• % implementers that publicly disclose expenditure on the health
program
IOM 2013 (30)
GHI Strategy
documents (31)
To what extent have financial
responsibilities been transferred
from donor to recipient?
• % of donor funds provided directly to recipient
• % donor contribution to program versus government funding
• Has there been any change in the budget?
• Has there been any change in funds arriving on time due to the
transition?
Bennett et al.
2011 (9)
GHI Strategy
documents (31)
36

Indicator
References c
To what extent has program
recipient secured adequate
funding to sustain program?
• % of budget allocated to the transitioning health program
• # of donors contributing to the program as percentage of target
• % gap between estimated annual program costs and resources
available
• Current distribution of program funding sources with projections
and end dates (i.e. donors, gov’t, fees)
• Grant applications submitted in last 12 months
• Total funds generated through successful grant applications in last
12 months
• Have any of the community groups and organizations secured
other sources of funding?
• % of budget that is generated through cost-generating activities
(e.g. arts and crafts products, direct appeal fundraising, etc.)
Bennett et al.
2011 (9)
Sarriot et al.
2009 (29)
Program
alignment
Relevant transition
archetype C
budgetary and financial alignment
between the donor and program
recipients?
• Overall budget and individual line items are reviewed and adjusted
• Alignment of donor and program recipient staffing requirements
• Alignment of donor and program recipient staff salaries
• During the transition period, have the program ever had any
problem with cash flows that has affected your operations?
• % donor funds provided directly to government
Bennett et al.
2011 (9)
GHI Strategy
documents (31)
To what extent are reporting
structures aligned between the
donor and program recipients?
• Has there been any change in the reporting format?
• % of indicators that match on donor and program recipient
reporting forms
• Alignment of donor and program recipient reporting frequency
• Alignment of donor and program recipient reporting literacy
requirements (i.e. pictorial vs. word-based forms)
• % of report that are complete upon submission
• % of reports that are submitted on time
Bennett et al.
2011 (9)
37

Indicator
References c
To what extent are service
delivery guidelines or
procurement aligned between
the donor and program
recipients?
• % of HCs with key stock-outs during transition
• Has there been any change in the supply chain of commodities
due to the transition?
• Is the implementer following the new service guidelines?
Bennett et al.
2011 (9)
To what extent do the program
monitoring and evaluation
systems align with the host
country’s M&E systems?
• % of indicators that match on program reporting forms with host
government HMIS
• Alignment of program and government reporting frequency
• % of indicators currently being reported to government HMIS
Bennett et al.
2011 (9)
a
Examples for program specific programs are indicated in parentheses.
b
Abbreviations: Community Health Workers (CHWs), Family Planning (FP), Health Centers (HC), HIV/AIDS (HIV), Health Monitoring Information System
(HMIS), Maternal, Newborn, Child Health (MNCH), Sexually-Transmitted Infections (STI), Tuberculosis (TB), Water, Sanitation and Hygiene (WASH).
Institutionalization
Relevant
construct
Guiding question Example Indicators a,b
Indicator
References d
38

Institutionalization
Relevant
construct
Indicator
References d
Institu-
tionalization
Relevant to all
transition
archetypes
To what extent are the
key features of the
original service
maintained in the
program post-transition
• Key feature X of the program continues post-transition (yes, no, partially)
Frequency with which key feature X of the program is
implemented/conducted post transition (always/frequently/never)
• Change in the frequency of the implementation/conduct of key feature X since
transition (same, lower frequency, increased frequency)
Bennett et al.
2011 (9)
Is there a regular budget
line and allocation to
support implementation
of this program?
• Necessary staff positions have been institutionalized and necessary posts
established (yes, no, partially)
• Budgets at national/district/facility level reflect funding necessary to support
transitioned program (yes, no, partially)
To what extent is the
program reflected in
routine norms and
guidelines?
• Government standard operating procedures reflect modalities of the
transitioned program (fully, partially, not at all)
• Clinical guidelines reflect modalities of the transitioned program (fully,
partially, not at all)
Is the program viewed as
a success by key
administrators and
program implementers?
• Key program administrators and implementers view the program as a success
(yes, no, partially)
a
b
c
Outcome indicators should be assessed pre- and post-transition to detect changesd
39

Sustained services and outcomes
Relevant
construct
Indicator
References d
Sustained
services
and
Outcomes
Relevant to
all transition
archetypes
To what extent is the
program recipient
controlling and managing
delivery of essential
program services?
• % of program services delivered through program recipient facilities
• % of program staff employed by program recipient
• # of referrals from program to patient support/advocacy groups
• # of patients referred to TB clinics (TB/HIV)
• Does the NGO/CBO have a linkage with government counseling and testing
services? (HIV)
Bennett et al.
2011 (9)
Scheirer &
Dearing 2011
(32)
How has the quality of
program services
changed?
• Prevalence of adequate iodine concentration (Nutrition)
• # of accurate messages shared during breastfeeding counseling sessions
(Nutrition/MNCH)
• % of mothers receiving at least one ANC visit by trained provider during last
pregnancy (MNCH)
• Changes in sensitivity of non-polio acute flaccid paralysis surveillance
(Immunization)
• % complete and correct HMIS monthly reports
• % of clients who are satisfied with the program’s services
Sarriot et al.
2009 (29)
Sebotsa et al.
2007 (33)
How has the coverage of
program services
changed before and after
transition?
• # of individuals in catchment area with access to water pumps and latrines
(WASH)
• Rates of voluntary medical male circumcision (HIV)
• Rates of XX vaccination (Immunization)
• # of CHWs serving district
• Ratio of CHWs to beneficiary population
• # of HCs providing service
• % of HCs open during regular hours yesterday
• What is the ratio of peer educators to high risk group? (HIV)
• What is the coverage of identified high risk groups with regular contact (two
contacts each month)? (HIV)
Bennett et al.
2011 (9)
Hoque et al.
1996 (34)
Sarriot et al.
2009 (29)
40

Sustained services and outcomes
Relevant
construct
Indicator
References d
How have key outcome
indicators and key health
outcome indicators
relating to your health
program changed?
• STI/HIV incidence (HIV)
• Incidence of unplanned pregnancy (FP)
• Incidence/prevalence of vaccine-preventable diseases (Immunization)
• Persistent and acute malnutrition prevalence in children (Nutrition/MNCH)
Jana et al. 2004
(35)
Sarriot et al.
2009 (29)
Scheirer &
Dearing 2011
(32)
How was the transition
experience overall?
• To what extent was the transition plan implemented as designed?
• How smooth was the transition experience?
• Has the overall program changed significantly as compared to pre-transition?
• Has the program recipient received support from the donor after the
transition?
Bennett et al.
2011 (9)
a
b
c
Outcome indicators should be assessed pre- and post-transition to detect changesd
6.3 Qualitative questions
Many of the guiding questions listed above could also be explored through qualitative methods. As such, we have included illustrative questions below that can
be used in semi-structured interviews or focus group discussions with key stakeholders who are either engaged in or affected in some way by transition.
Several of these questions explore multiple dimensions at once so they have only been divided around timing, e.g. before, during or after transition. As with
the quantitative indicators, these questions draw from past experiences with evaluations of transition (7,9) as well as our collective experience.
41

Table 5 – Qualitative questions for M&E
Transition
period
Illustrative question
Pre-transition
1. How was the decision to end donor support made?
a. Were the program and recipient organization involved in this decision? If not, why not?
b. How was this decision communicated to the program and the recipient organization?
2. How was the transition planned?
a. Which actors were important in planning transition? Donor’s role? Recipient’s role? Program staff role?
i. Probe on role of donor, recipient, program staff, beneficiaries, government, others actors (e.g.
development partners, civil society, etc)
ii. Which individuals or groups were in charge of making the final decisions about the plans?
iii. Who should have been involved in the planning process that was not included? Why?
iv. For those actors were not involved in the planning process, were these plans communicated to them? If
yes, how?
b. Was the transition plan written into any document? If so, was the document ever updated?
3. What did specific activities did you/your organization conduct to prepare and plan for transition?
a. Were additional staff hired to manage transition or post-transition activities? What was their role?
b. How have capacity needs been assessed?
c. What plans are in place for addressing capacity needs?
4. What were the main concerns or risks that were identified during transition planning?
a. What type of evidence was used to determine these risks and opportunities?
b. Which individuals or groups were in charge of identifying this evidence?
c. What plans were made to mitigate or eliminate these risks?
5. What aspects of the transition process was there least concern about?
a. Were there any aspects that were seen as an opportunity to bring forth programmatic improvement?
42

Transition
period
Transition
preparation
1. How long before transition did the preparations for transition start?
a. Was this a sufficient amount of time? If not, why not?
2. How well was the program working prior to transition?
a. Were there any implementation issues that had not been addressed prior to the start of transition?
b. How did the transition affect these existing issues?
3. Who has played key roles in the transition process?
a. What was their role? How did this change over time?
b. Have there been any stakeholders that have not played a role in transition but should have? Why?
4. How was the decision to transition communicated to local stakeholders and program beneficiaries?
a. How have local stakeholders and program beneficiaries perceived the decision to transition?
b. When you first learned about transition, how did you think that it would affect service delivery?
5. How was the decision to end donor support perceived by other program supporters, including other development
partners?
a. How did transition affect the relationship with other program partners? Did it increase their interest in
transitioning as well?
6. What policies are in place to support the objectives of the program?
a. Probe about recipient organization policies; local/state/federal policies
b. Are there existing policies that threaten the future of the program? Why?
7. How did the recipient and other actors ensure that sufficient financial resources would be available after the end of donor
support?
a. What arrangements were made to raise sufficient funds?
i. Probe about: short-term, medium, and long-term budgets?
ii. What gaps in the budget were the most difficult ones to fill?
iii. What kind of trade-offs was considered?
iv. What do you think about the adequacy of these arrangements?
b. What types of financial projections were conducted? How helpful were they?
8. How was the transition process coordinated?
a. Who was in charge of coordinating donor support before the transition process began?
i. What was their role during transition?
ii. What should have been their role after transition?
iii. Should these coordination mechanisms have been sustained?
9. What opportunities have there been to make changes to transition plans and activities based on ongoing experiences?
a. Are there regular meetings between stakeholders to discuss the transition process? What has emerged from
these meetings?
b. How have challenges identified during implementation of the transition plan been addressed?
43

Transition
period
Post-
transition/
Institutionali-
zation
1. How effective was the transition process?
a. How well did the transition plan work for implementing transition?
b. What worked well?
c. What could have been improved? Any missed opportunities?
2. How do the systems and structures of the program function after transition?
a. What changes have taken place as a result of transition? Have these been positive or negative changes? Why?
b. Probe about service delivery; supply availability and logistical management; staffing; finances; ownership and
accountability; role of external actors; regional differences, if applicable
3. How have program objectives been sustained after the completion of donor support?
a. Probe about supply and demand-side features of the program.
b. If the indicators have not been met, probe about why.
4. In the future, how do you think the recipient will handle the adaptation and/or scale-up of existing programs?
a. Probe about ownership and accountability
b. Probe about local capacity
c. Probe about how the program has evolved since transition?
5. What discussions have taken place about institutional sustainability?
6. In the future, how do you think the donor/recipient will approach a similar transition?
a. What should be done differently in a similar transition?
44

7. TRANSITION CHALLENGES THAT AFFECT M&E
The transition process is rarely easy as organizational change is occurring at various levels in a health
program. Thus, introducing the monitoring and evaluating as an additional component in the process of
transition, could be met with skepticism and resistance. The transition process involves numerous
stakeholders during demanding times. Engaging stakeholders early to support these additional efforts to
monitor and evaluate transition will take a conscious and active effort.
In Table 6, we summarize potential challenges and solutions, as we reflect upon experiences of transition
and M&E of transition to identify common challenges, and provide guidance on potential strategies for
mitigating them while engaging in M&E of transition. Knowledge of these transition challenges may be
most useful to an independent evaluator involved in the M&E of transition, but can benefit other parties,
such as donor coordination groups, involved in transition as the process proceeds.
Table 6: Challenges in M&E of transition and potential solutions
Challenges in M&E of transition and potential solutions
Challenge Potential solutions
M&E
Lack of buy in from partners: if program
recipient doesn’t buy in to M&E of
transition, evaluators are stuck because
they can’t get post-transition data to
assess service delivery and impacts and
how those might have changed due
to/after transition
• Get program recipient involved in RFP for M&E and
get them to pay for part of it.
• Ensure sound communications throughout the
process, and that the benefits of monitoring and
evaluating transition are understood by
stakeholders at all levels (not only high level
managers, but also front line workers) and in all
organizations
Political sensitivities: evaluators may not
be granted access to key documents or
organizations because stakeholders are
afraid to appear less than stellar
• Presentation of neutral results to-date can help
reassure skittish stakeholders. Also, promising
confidentiality and anonymity.
Resistance from within the program
recipient organization: Additional
assigned work from M&E of transition or
shifts in organizational priorities due to
transition may create resentment and
negativity towards transition (e.g., a shift
from providing services for the poor to
an increasing focus on being cost-effective
with services to ensure sustainability).
• In these instances, communication from higher
levels on the strategic priorities of the organization,
and how new changes are fitting into the
organization’s overall mission are crucial to
sustaining support for the organization’s work in
time of transition.
45

Data quality: when possible, it’s good to
use/build from whatever existing data
programs are already collecting (e.g.
service delivery) but sometimes data
inconsistencies may exist in program data
and reported data.
• Build on existing systems where possible; maintain
close connections with managers of reporting
systems to ensure transparency
Creating change from M&E data:
Recommendations from M&E data and
reports may prove unpopular and face
inertia as they may require additional
work from the program recipients.
• Put mechanisms in place to discuss data, explore
recommendations, & delegate responsibility for
implementing them. Follow up on progress. There
should be shared responsibility for utilizing the data
so as to create positive change towards the
organization’s mission.
Attaining financial and human resources
for (the M&E of) transition: Though
resources for managing the transition
process itself may be easier to obtain,
dedicated financial and human resources
for transition monitoring may be more
difficult to come across during a time of
already financial hardship.
• Advocate early during the pre-transition process
(and even as early as the program planning) to have
budgetary allocations for the M&E of transition
Transition-
related
challenges
that would
affect M&E
of
transition
Timing and timelines: Often funding
decisions are made at very high levels –
i.e. USG – and this does not give
sufficient time for a transition process to
be planned.
• Try to make transition part of the picture as early
as the program planning stages.
Break down in relationship between
donor and program recipients:
sometimes these relationships go
seriously awry
• Evaluation team develops strong, independent
relationships with various stakeholders. Hopefully
presenting results to get more buy-in will help
smooth the way too.
Managing expectations: programs after
the transition may not deliver the same
quality and quantity of services, and in the
same manner as before transition
• Communicate with stakeholders at all levels,
including program staff, health workers, and
program beneficiaries of the transition plans,
reasons for transition as well as any potential
benefits.
46

Coordination with other donors running
related or connected programs
• Map involvement/coverage of services from other
donors and assess extent of influences on and
linkages to program to be transitioned.
• Written communication to other donors about the
transition and the specific parameters of transition,
followed-up by on-going discussion in donor
coordination meetings.
• If applicable, bi-lateral donor coordination
discussions and coordination to leverage strengths
of each donor.
47

ANNEX A: FURTHER READING
Avahan Transition
• Bennett S, Singh S, Ozawa S, Tran N, Kang JS. Sustainability of donor programs:
evaluating and informing the transition of a large HIV prevention program in India
to local ownership. Glob Health Action. 2011;4. doi: 10.3402/gha.v4i0.7360.
GAVI Transition
• Curatio International Foundation. Final Evaluation of GAVI Alliance’s Support to Bosnia
and Herzegovina. 2014.
Family Planning Graduations in Latin America
• Bertand, J. USAID Graduation from Family Planning Assistance: Implications for Latin
America. New Orleans, LA: Population Institute; 2011.
• Shen AK, Farrel MM, Vandenbroucke MF, Fox E, Pablos-Mendez A. Applying lessons learned
from the USAID family planning graduation experience to the GAVI graduation
process. Health Policy Plan. 2014. doi: 10.1093/heapol/czu045.
Other readings on transition and sustainability
• Gardner BA, Greenblott K, Joubert E. What We Know About Exit Strategies Practical
Guidance For Developing Exit Strategies in the Field. C-Safe Regional Learning Space
Initiative; 2005.
• Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability
science: an integrated approach for health-programme planning. Lancet. 2008 Nov
1;372(9649):1579–89.
• Levinger B, McLeod J. Hello I must be going: Ensuring quality services and sustainable
benefits through well-designed exit strategies. Newton, MA: Center for Organizational
Learning and Development; 2002.
• Brundage SC, Bellamy WM, Lamptey P, Morrison JS, Bliss KE, Cooke JG. Terra Nova: How to
achieve a successful PEPFAR transition in South Africa. Washington, DC: USAID; 2011.
51

ANNEX B: REFERENCES
1. Bossert TJ. Can they get along without us? Sustainability of donor-supported health projects in
Central America and Africa. Soc Sci Med. 1990;30(9):1015–23.
2. Bertrand JT. USAID graduation from family planning assistance : Implications for Latin America. New
Orleans, LA: Population Institute; 2011.
3. Office of Global AIDS Coordinator. The United States President’s Emergency Plan for AIDS Relief -
Partnership Frameworks [Internet]. 2008 [cited 2014 Sep 01]. Available from:
http://www.pepfar.gov/about/strategy/global_context/133382.htm.
4. Office of Global AIDS Coordinator. Putting Country Ownership into Action [Internet]. 2012 [cited
2014 Sept 01]; Available from: http://www.pepfar.gov/countries/frameworks.
5. McNeil D. AIDS Progress in South Africa Is in Peril [Internet]. New York Times. 2014 [cited 2014
Oct 10]. Available from: http://www.nytimes.com/2014/08/26/health/aids-south-africa-success-
pepfar.html.
6. Sgaier SK, Ramakrishnan A, Dhingra N, Wadhwani A, Alexander A, Bennett S, et al. How the
Avahan HIV prevention program transitioned from the Gates Foundation to the government of
India. Health Aff (Millwood). 2013 Jul;32(7):1265-73
7. Curatio International Foundation. Final Evaluation of GAVI Alliance’s Support to Bosnia and
Herzegovina. 2014.
8. Chee G, Xie Z, Nakhimovsky S. Evaluation of GAVI- Government of China Hepatitis B. Geneva: Abt
Associates; 2012.
9. Bennett S, Singh S, Ozawa S, Tran N, Kang JS. Sustainability of donor programs: evaluating and
informing the transition of a large HIV prevention program in India to local ownership. Glob Health
Action. 2011 Jan;4:1–9.
10. ICF International. Monitoring the transition of HIV programs to country ownership. 2014.
11. Saxenian H, Hecht R, Kaddar M, Schmitt S, Ryckman T, Cornejo S. Overcoming challenges to
sustainable immunization financing: early experiences from GAVI graduating countries. Health Policy
Plan. 2014 Feb 8. [Epub ahead of print]
12. Rose J, Deussom R, Petrozzola H, Blyth K. Transitioning health care worker support: A resource
framework and lessons learned from the PEPFAR experience. In: Proceedings of the 2014
International AIDS Conference; 2014 Jul 20-25; Melbourne, Australia. [cited 2014 Oct 10]. Available
from: http://pag.aids2014.org/Abstracts.aspx?AID=3209
53

13. Levinger B, McLeod J. Hello I must be going: Ensuring quality services and sustainable benefits
through well-designed exit strategies. Newton, MA; 2002 p. 27.
14. Buse K, Martin G. AIDS: ushering in a new era of shared responsibility for global health. Global
Health. Globalization and Health; 2012 Jan;8(1):26.
15. Gardner BA, Greenblott K, Joubert E. What We Know About Exit Strategies Practical Guidance For
Developing Exit Strategies in the Field. C-Safe Regional Learning Space Initiative; 2005.
16. Canadian Public Health Association. Sustainability and Equity : Primary Health Care in Developing
Countries. Canadian Public Health Association; 1990.
17. Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability science: an
integrated approach for health-programme planning. Lancet. 2008 Nov 1;372(9649):1579–89.
18. Goldberg J, Bryant M. Country ownership and capacity building: the next buzzwords in health
systems strengthening or a truly new approach to development? BMC Public Health. 2012
Jan;12:531.
19. Potter C. Systemic capacity building: a hierarchy of needs. Health Policy Plan. 2004 Sep 1;19(5):336–
45.
20. Brundage SC, Bellamy WM, Lamptey P, Morrison JS, Bliss KE, Cooke JG. Terra Nova: How to
achieve a successful PEPFAR transition in South Africa. Washington, DC: Center for Strategic and
International Studies; 2011.
21. Johnston T. Supporting A Healthy Transition Lessons from Early World Bank Experience in Eastern
Europe. Washington, DC: World Bank; 2002.
22. Gage AJ, Ali D, Suzuki C. A Guide for Monitoring and Evaluating Child Health Programs. New York:
UNICEF; 2005.
23. Finn, T. A guide for monitoring and evaluating population-health-environment programs.
Washington, DC: USAID; 2007,
24. Pisani, E. National AIDS programmes: A guide to monitoring and evaluation. Geneva: UNAIDS;
2000.
25. De Savigny D, Adam T. Systems thinking for health systems strengthening. Geneva: World Health
Organization; 2009.
26. Kelley E, Hurst J. Health care quality indicators project: conceptual framework paper. Paris: OECD;
2006.
27. Johnson, Knowlton, et al. Building capacity and sustainable prevention innovations: A sustainability
planning model. Evaluation and Program Planning 2004. 27(2):135-149.
54

28. Fox AM, Balarajan Y, Cheng C, Reich MR. Measuring political commitment and opportunities to
advance food and nutrition security: piloting a rapid assessment tool. Health Policy Plan. (2014) doi:
10.1093/heapol/czu035
29. Sarriot, E., Ricca, J., Ryan, L., Basnet, J., & Arscott‐Mills, S. (2009). Measuring sustainability as a
programming tool for health sector investments: report from a pilot sustainability assessment in five
Nepalese health districts. The International journal of health planning and management, 24(4):326-
350.
30. Black RE, Auerbach JD, Bassett MT, Brookbeyer R, Dare L, Ezeh AC, et al. Evaluation of PEPFAR
Committee on the Outcome and Impact Evaluation of Global HIV / AIDS Programs. Washington,
DC: National Academy of Sciences; 2013. Chapter 10, Progress Towards Transitioning to a
Sustainable Response in Partner Countries; p. 1-47.
31. Knowledge-driven international development. Indicators/metrics from GHI strategies [Internet].
2014 [cited 2014 Oct 10] KDID; Available from:
http://kdid.org/sites/kdid/files/resource/files/Indicators from GHI Strategies.pdf
32. Scheirer, M. A., & Dearing, J. W. An agenda for research on the sustainability of public health
programs. Am J Public Health 2011. 101(11):2059.
33. Sebotsa ML, Dannhauser A, Jooste PL, Joubert G. Assessment of the sustainability of the iodine-
deficiency disorders control program in Lesotho. Food Nutr Bull 2007; 28: 337–47
34. Hoque BA, Juncker T, Sack RB, Ali M, Aziz KM. Sustainability of a water, sanitation and hygiene
education project in rural Bangladesh: a 5-year follow-up. Bull World Health Organ 1996; 74:431–
437.
35. Jana S, Basu I, Rotheram-Borus MJ, Newman PA. The Sonagachi Project: a sustainable community
intervention program. AIDS Educ Prev 2004; 16:405–14.
36. GAVI Alliance. GAVI Alliance Graduation Policy. GAVI Alliance: Washington, DC; 2011.
55

Guide for the Monitoring and Evaluation of the Transition of Health Programs

More Related Content

What's hot (6)

Similar to Guide for the Monitoring and Evaluation of the Transition of Health Programs (20)

More from HFG Project (20)

Recently uploaded (20)

Guide for the Monitoring and Evaluation of the Transition of Health Programs