Evidence Diagnostic - Thinking & Working Politically

Understanding the demand and use of
evidence through a ‘political economy +’
approach: the SEDI experience in Ghana,
Pakistan, and Uganda
SEDI Learning Brief 1
Using evidence diagnostics to think and
work politically: SEDI’s experience

2 SEDI Learning Brief 5
Acronyms
AAA Authority, acceptance, ability model
BCURE Building Capacity to Use Research Evidence
DFID UK Department for International Development (now merged into FCDO)
EIDM Evidence-informed decision-making
EPRC Economic Policy Research Centre (Uganda)
FCDO UK Foreign, Commonwealth & Development Office (formerly DFID)
INASP International Network for Advancing Science and Policy
M&E Monitoring and evaluation
MDA Ministries, departments and agencies
NDP National Development Plan
NTC National Tariff Commission (Pakistan)
OPM Oxford Policy Management
PEA Political economy analysis
PEA+ Adapted political economy analysis
SDPI Sustainable Development Policy Institution (Pakistan)
SEDI Strengthening the Use of Evidence for Development Impact

3
Key
messages

„ ‘Working politically’ requires practitioners to
have trusted working relationships with
government which take time to cultivate and
in SEDI’s case predated the programme.

„ Relationships can be both expressed and
enabled by co-creative approaches, but
this way of working can take time to
socialise as it remains outside the norm for
some in the development sector.

„ Authority, acceptance and ability (AAA)
enabled SEDI to begin to understand the
appetite of and space within government
agencies to partner with the programme and
further explore their use of evidence through
an evidence diagnostic.


„ SEDI and its government partners built
on the AAA analysis using the Context
Matters Framework to support the
move from ‘thinking politically’ in the
project’s analysis phase towards ‘working
politically’ in implementation.

„ To be agile and adaptive, delivery teams
need to make specific methodological
choices about their scope and ways of
working. This includes making compromises
in order to preserve government ownership
of interventions.

„ The COVID-19 pandemic caused a
radical and unforeseen shift in ways of
working for all partners involved in the
diagnostics; the team was able to adapt the
approach and transition to a virtual working
environment COVID-19 but there were
implications for the participatory nature of
the diagnostic.

5
Introduction
Evidence use in policy is fundamentally
intertwined with how government works – its
cultures, processes and structures, both formal
and informal. Those working to strengthen the
use of evidence therefore need to understand
how to navigate processes of government
reform and the opportunities that this presents.
Participatory diagnostic exercises can be
used to build this understanding by providing
the space and the tools to help government
departments and their partners to look at how
they already use evidence and what they
could be doing differently within the context of
their organisational culture and systems. This
establishes strong foundations for co-ownership
and helps partners to jointly define problems,
which in turn underpins the design and delivery
of interventions.

This brief shares reflections from two evidence
diagnostic exercises (one in Pakistan and the
other in Uganda) undertaken as part of the
Strengthening Evidence Use for Development
Impact (SEDI) programme, funded by the UK
Foreign, Commonwealth & Development Office.
For SEDI, evidence diagnostics bridge the
gap between political economy analysis and
programme implementation, providing one of
the first opportunities to put into practice the
approach of ‘thinking and working politically’
(TWP) – which emphasises political insight and
analysis, responsiveness to context, and flexible
and adaptive design and implementation.
Located at the intersection between evidence-
informed decision-making (EIDM) and
broader governance approaches that aim to
fortify public institutions and improve public
sector performance and accountability, the
SEDI experience may offer insights for both
those working on organisational reform and
institutional change in a public sector context,
and those working in knowledge and research
systems. For those in the research and
knowledge sector, SEDI’s experience shows
how methods originating from governance
and state capability approaches can build
understanding of organisational and institutional
reform in government and therefore enhance
understanding of how evidence is used. For
those operating in the governance sector,
SEDI’s experience illustrates that evidence is a
key element of strengthening public institutions’
performance and accountability.
In both Pakistan and Uganda, the diagnostic
exercises were carried out by joint teams
from the SEDI programme and from selected
government agencies in each country. Having
collaboratively adapted and tailored the
chosen diagnostic tool – Context Matters –
these joint teams carried out participatory data
collection activities to understand the barriers
and opportunities for strengthening evidence
within the respective government agencies and
identified priority issues for the SEDI programme
(and potential solutions it could explore).
The exercises were led by the Sustainable
Development Policy Institution (SDPI) in
Pakistan, in partnership with the National Tariff
Commission, and the Economic Policy Research
Centre (EPRC) in Uganda, in partnership with
the Office of the Prime Minister. SEDI’s capacity
development and brokering workstreams, led by
INASP and Oxford Policy Management (OPM)
respectively, provided advisory support.
In Uganda, the diagnostic focused on the new
National Development Plan (NDP III), which
involved several different government agencies;
in Pakistan, it was focused at the organisational
level of the National Tariff Commission, rather
than on a multiagency planning process.
Political economy analysis undertaken in the first
year of SEDI had identified both agencies as
entry points for the programme.

7
Why carry out
an evidence
diagnostic?
Building on political
economy analysis
In its first year, the SEDI programme embarked
on an analysis phase, during which it conducted
an adapted political economy analysis (PEA+)
in each of its three focus countries: Ghana,
Pakistan and Uganda. The aim of this analysis
was to ‘set the scene’ and ensure that decisions
on entry points for SEDI’s work were informed
by an understanding of both the broad political
economy context and the likely buy-in and
interest among potential partner agencies in the
sector.1

The AAA analysis used as part of the PEA+
aimed to provide a preliminary assessment
of government agency ‘readiness’ to engage
with the SEDI programme to avoid common
challenges such as isomorphic mimicry and
premature load bearing.2
SEDI PROGRAMME LAUNCHED
Analysis phase begins (8 months)
INCEPTION PHASE BEGINS (10 MONTHS)
Uganda diagnostic starts from October
IMPLEMENTATION PHASE BEGINS
Pakistan diagnostic with NTC begins in March
July
2019
March
2020
January
2021
Figure 1: SEDI timeline
originally to 2024

9
SEDI completed the analysis phase and entered
its inception phase, just as the COVID-19
pandemic began (Figure 1 [timeline]). In
response to the pandemic, SEDI consortium
chose to adapt, delaying the diagnostics until
the end of the inception phase and prioritising
immediate ways to support government
agencies to respond to the crisis. (This pivot
was, in itself, illustrative of a ‘thinking and
working politically’ approach – see Box 1.) The
evidence diagnostics began in the inception
phase, aiming to lay the groundwork for the
government partnerships that were to be
the core pillars of the SEDI programme and
inform the design of interventions that it would
undertake. In this way the diagnostics supported
the transition from ‘thinking politically’ (the
analysis phase) to ‘working politically’ (the
implementation phase), as illustrated in Figure 2.
The SEDI programme had envisaged that
diagnostics would have been conducted in in
all three countries by the end of the inception
phase, providing a set of priority evidence
issues for the programme to work on in each
agency (‘problems’) and possible interventions
to address these (‘solutions’), which would then
be piloted (Figure 2). However, the delays in
the inception phase due to COVID-19 and the
subsequent closure of the SEDI programme just
as implementation began meant that (1) only two
of the three focus countries undertook evidence
diagnostics and (2) the SEDI programme ended
immediately after the diagnostic process rather
than continuing to support the change plans that
had emerged.
Box 1: What does ‘thinking and working politically’ mean to SEDI?
Limited success – or outright failure – in development programming has been linked to
insufficient attention being given to contextual issues, such as political, socioeconomic, and
cultural systems and prevailing norms. While there are no set criteria or methods, some
core principles have emerged about what it means to think and work in a more contextually
aware way (‘thinking and working politically’ – or TWP). These are that international
development approaches need to be:

„ problem-driven rather than intervention-driven;

„ grounded in contextual realities;

„ locally led and owned;

„ adaptive, flexible, iterative, and often entrepreneurial;

„ more open to risk and failure;

„ staffed with skilled and experienced people who are comfortable with the political nature
of development and have deeply rooted contextual knowledge and networks they can
tap into;

„ anchored in international development actors’ roles as enablers, brokers, and
convenors of locally led reform processes, rather than simply as funders, directors, or
implementers; and focused on changing behaviours, not just formal rules.
Source: Adapted from Box 1 in Shaxson et al. (2021).

Box 2: Lessons from the Building Capacity to Use Research Evidence
(BCURE) programme in thinking and working politically

„ Political economy factors are critical elements shaping evidence use in policy.

„ BCURE projects were not required to carry out political economy analyses at the
inception of the programme, although implementers strengthened their understanding of
these dynamics during delivery.

„ Projects should have looked beyond ‘face-value’ statements of buy-in and interest from
partner government agencies to explore deeper power and political dynamics within
government partner organisations.

„ Interventions had greater success where they responded to existing entry points or
windows of opportunity in a government agencies or sectors.

„ Identifying and working with champions was also a key success factor in navigating
political dynamics.

„ DFID’s contracting model, which was structured around milestones, made it challenging
for implementing partners to respond in an agile and flexible way to shifting political
dynamics.
Source: Vogel and Punton (2018)..
Figure 2: From analysis to implementation
ANALYSIS
REPORTS

„ Big picture overview of
the wider context

„ Initial identification
of key government
agencies to target

„ ‘Light touch’ authority,
acceptance and ability
(AAA) analysis of
priority organisations
ORGANISATIONAL
DIAGNOSTICS

„ Co-designing with
government an in-depth
participatory process for
diagnostics, incorporating
a gender equity and social
inclusion lens

„ Analysing internal
structures, processes,
culture, ways of working
etc. (includes capacity
development (CD),
brokering relationship
building)
Outcome: Key issues
co‑defined and areas for
intervention prioritised
INTERVENTIONS
TOOLBOX

„ Creating menu of possible
CD and brokering
interventions to address the
diagnosed issues

„ Developing links to:

» Existing evidence base for
interventions

» Theory of change (incl.
relationship between
intervention and outcome)

» CD and brokering learning
questions (eg, on GESI)
‘Living tool’ updated with
learning from implementation
INTERVENTION
PILOTS

„ Designing and adapting
combination of
interventions at different
levels to be trialled to meet
the diagnosed issues

„ May focus on CD or
brokering, or combine

„ Learning feeds into
refinement of Toolbox
PROBLEMS SOLUTIONS
All underpinned by & feeding into SEDI theory of change through an ongoing iterative process

11
The SEDI programme’s approach to TWP
was in many ways a direct response to
the evaluation findings from the Building
Capacity to Use Research Evidence (BCURE)
programme (Box 2), which was funded by the
UK Department for International Development
(DFID) from 2013 to 2017. However, SEDI
had a much more flexible programme design
structure than BCURE, with no fixed activity
milestones with government partners set at the
outset. To decide which activities to pilot the
programme therefore sought to build a deep
understanding of organisational dynamics
in government agencies. This would ensure
intentional, strategic operational choices were
informed by an understanding of political and
power dynamics, and therefore improve the
potential for the interventions’ sustainability and
effectiveness.
Strong and trusted relationships with
government agencies enabled and contributed
to this understanding, and would facilitate the
close collaboration needed to design and pilot
interventions over the course of the programme.
The SEDI programme did not see evidence
diagnostics as a purely analytical or extractive
research process but as a practical, operational
way to transition the programme from the
analysis phase into implementation. The role
of the SEDI team was to facilitate a process
through which a public agency could begin
to identify and understand the key factors that
affect how they are using evidence and priority
areas to address. The diagnostics also had
particular relevance to ensure a high-quality
and evidence-informed approach to SEDI’s two
technical workstreams: capacity development
and evidence brokering (see Box 3).
Box 3: SEDI’s technical workstreams
The SEDI programme had two interlinked technical strands of work:
Capacity development in the SEDI programme focused on strengthening capacities at
individual level (i.e. among public servants) and organisational level (i.e. within government
agencies) to systematically use evidence in policymaking. ‘Capacity’ is understood to
encompass both ‘hard/technical aspects’, such as skills or organisational processes,
and ‘soft’ aspects such as organisational culture. INASP led the capacity development
workstream.
Evidence brokering in the SEDI programme has focused on two aspects: evidence
translation and relationship brokering. Evidence translation involves making evidence
accessible and useful for decision-making; and relationship brokering focuses on building
trust-based relationships between the generators and the users of evidence. OPM led the
brokering workstream.

Identifying interventions to pilot under SEDI
Practitioners seeking to strengthen evidence
use in policymaking use a wide range of
interventions.3
To decide which interventions
to pilot in SEDI and how to combine these
at multiple levels, the programme needed to
build a joint understanding with colleagues
in government agencies about where the
problems within those agencies lay and which
of these might be appropriate to tackle. The
programme sought to avoid an approach
driven by predefined interventions and rather
aimed to ensure that intervention design and
selection was rooted in a strong and shared
understanding of the cause of the problems
identified.
Specifically, the evidence diagnostics aimed to:

„ identify appropriate entry points within the
agency/ies – for instance, particular units or
champions that would be most strategic to
engage (understanding that motivation to
focus on evidence use, capacity needs, and
‘on paper’ responsibility to handle evidence
may not all exist in the same unit);

„ enable co-definition with government
partners of the roots of the identified
problems in order to build the groundwork for
solutions to be fully owned;

„ identify priority areas for action where SEDI is
best placed to support;

„ inform the design and timing of interventions
using a technical toolbox developed by
the SEDI programme to be a ‘living tool’
or menu of possible interventions that
could be adapted and used to increase
the instrumental and embedded use of
evidence.4

13
Approach and
methodology
The SEDI programme used the Context Matters
Framework, a participatory diagnostic tool initially
developed by Politics & Ideas5
in collaboration
with INASP as part of the BCURE VakaYiko project
(Weyrauch et al., 2016).6
Combining elements of
political economy analysis, knowledge systems
analysis and organisational assessment, Context
Matters identified six dimensions of context affecting
evidence use in policy: (1) macro context; (2) intra-
and interrelationships; and the agency’s (3) culture,
(4) capacity, (5) management and processes,
and (6) other resources. The aim is to help users
identify a feasible, realistic and politically viable
change pathway for greater evidence use within a
government agency (Box 4; Figure 3). Knowledge
about each of these dimensions is collected through
document review, a series of workshops, focus

group discussions and interviews, and is used
to identify priority areas for action and a change
plan or strategy to implement these.
Box 4: Guiding principles for the Context Matters Framework

„ It is not primarily designed as a research tool but is a guide for a facilitated reflective
exercise, where the participatory process (carried out by a joint team from the project
and the partnering government agency) is as or more important than the final product
or report.

„ It emphasises the value of co-identifying problems and priorities with government
partners before identifying solutions.

„ It focuses on both the visible factors affecting evidence use ( ‘hard’ or ‘technical’ aspects
such as infrastructure, official plans and structures) and invisible factors (largely related
to organisational culture, narratives, beliefs and openness to change).

„ It starts from a systemic point of view – seeing problems in context and actively
exploring the links and relationships between different components and stakeholders.

„ It does not aim to be fully comprehensive and explore all areas in equal depth, but to
provide a way to ‘zero in’ on specific priority areas from a broader perspective.
WORKSHOPS 1 & 2
Familiarisation and adaptation
of the diagnostic tool
(Context Matters)
WORKSHOPS 3 & 4
Plan and design the
approach
DIAGNOSTIC IMPLEMENTATION
Workshops with government agencies
Interviews/focus groups
Identification of priority areas for action
Change plan
Synthesis of findings (report or
presentation)
Diagnostic design

15
Figure 3: Context Matters Framework7
The SEDI programme adapted the Framework
to respond to the contextual picture identified
by the PEAs, learning from previous evidence
diagnostic exercises8
and the needs and
priorities of the respective government agencies.
In using both the ‘AAA’ framing and then the
Context Matters Framework, SEDI chose to work
with existing tools, and adapt and tailor these to
the programme and country contexts. This was
seen as an opportunity capitalise on existing
knowledge and expertise that the tools captured
and offered all partners the opportunity to
strengthen capacity around these approaches
and the knowledge and skills they build.
SEDI partners found it important to establish
parameters for the key dimensions to be
included in the diagnostics, providing guidance
on what is meant by ‘evidence’, ‘evidence
ecosystem’, and ‘(gender) equality and social
inclusion’ (GESI)9
.
04.
ORGANIZATIONAL
CAPACITY
03.
CULTURE
MEDIA
STATE AGENCY
STATE AGENCY
STATE AGENCY
STATE
DONORS
PRIVATE
COMPANIES
RESEARCH
ORGANIZATIONS
CITIZENS
02.
INTRA & INTER
RELATIONSHIPS
05.
MANAGEMENT
& PROCESSES
06.
OTHER
RESOURCES
01. MACRO CONTEXT

Uganda
In Uganda, the evidence diagnostic focused
on the Development Plan Implementation
Programme (‘Programme 18’), one of 18
priority programmes under the country’s Third
National Development Plan (NDP III), which
aims to improve household income and quality
of life for Ugandans. The 18 programmes were
created to address the persistent challenges
resulting from uncoordinated planning, weak
harmonisation, limited programme sequencing
and poor links between outcomes and
outputs. Each programme has a set of clearly
articulated results, objectives and interventions.
Programme 18 focuses on increasing
efficiency and effectiveness in the NDP III’s
implementation and is at the heart of delivering
the NDP III; one of its key targets is to ensure at
least 80% achievement of the NDP III.
EPRC, the diagnostic leads in Uganda, already
had an existing relationship with the Monitoring
and Evaluation (M&E) team at Office of the
Box 5: Action-oriented diagnostic for NDP III Programme 18, Uganda
(October 2020 – June 2021)
The main partner was the M&E Directorate in the Office of the Prime Minister. The Prime
Minister, as leader of government business and the overall coordinator, is responsible
for leadership of NDP III implementation. The Office of the Prime Minister also houses a
secretariat which consolidates NDP III progress reports from Ministries that are leading
the implementation of individual programmes. The Directorate’s official mandate includes
coordinating monitoring and evaluation of government policies and programmes across
MDAs and, as part of this, ensuring that evidence is available to inform decision-making.
Through the Directorate, SEDI convened four apex agencies responsible for planning,
monitoring, evaluating and budgeting: Office of the Prime Minister; Ministry of Finance,
Planning and Economic Development; Office of the President; and National Planning
Authority. Five other MDAs were also included as they are critical to the functions played by
Programme 18:

„ Ministry of Local Government, to establish the use of evidence in fostering grassroots
development and service delivery, as well as generation of quality of administrative data;

„ Ministry of Public Service, to address issues of human resource capacity needs;

„ Ministry of Science, Technology and Innovation, as regulators of research in the country;

„ Uganda Bureau of Statistics, as the principal data collecting, processing, analysing
and disseminating agency responsible for coordinating and supervising the National
Statistical System.

„ Ministry of Information Communication Technology and National Guidance, as the body
responsible for ICT legal and regulatory environment, secure ICT access and usage for
all and increased awareness and citizen participation in government programmes.

17
Prime Minister, which served as a link and
entry point to building relationships with other
ministries, departments and agencies within
Programme 18.
The first step was to adapt the diagnostic
methodology for a cluster of agencies that
work together under Programme 18 and
involved intensive stakeholder identification,
coordination and communication to avoid having
single diagnostic pieces that didn’t relate to
each other. This was followed by an in-depth
stakeholder mapping exercise to analyse the
landscape and unravel patterns in evidence
production and use. Given the multiple players
involved in Programme 18, the Context Matters
Framework dimension on interrelationships
(coordination, evidence flow and channels
of interaction) had considerable weight and
required special focus.
A design phase to familiarise participants with
the diagnostic framework, to agree roles and
responsibilities, and to identify other ministries,
departments and agencies (MDAs) that could
take part in the diagnostic process followed.
This was longer than originally expected, not
only due to the pandemic but also because of
the level of adaptation required to carry out a
diagnostic of a whole programme (as opposed
to a single agency, which would be much
simpler). This stage which included workshops
with the M&E Directorate in the Office of the
Prime Minister was primarily conducted online.
Finally, the implementation phase benefited
from a face-to-face approach as COVID-19
restrictions started to ease. EPRC led the
workshops and data collection activities, with the
close support of SEDI team in frequent online
meetings.
Pakistan
In Pakistan, the evidence diagnostic focused
on the National Tariff Commission (NTC), the
government’s autonomous investigation authority
on trade and tariff matters. As in Uganda, the
Context Matters Framework was adapted to
this diagnostic focus, building on extensive
work undertaken during the SEDI programme’s
analysis phase to build a coherent picture
of the politics of evidence use in Pakistan’s
key sectors. SDPI’s involvement in the Prime
Minister’s Economic Advisory Council as well as
other prior collaborations had helped deepen the
relationship with key institutions within the trade
space, including the National Tariff Commission.
The NTC wanted to strengthen its role as a think
tank for the government and to improve how
it handled and communicated evidence to be
more effective in informing policy.
Given this objective, the diagnostic team
placed particular emphasis on the intra- and
interrelationships dimension of the Context
Matters Framework to capture feedback from
external stakeholders (other government and
non-government actors) on how the NTC
operates. The diagnostic’s planning and design
stage was conducted primarily online with
focal points from the NTC while the delivery
of workshops was staggered: it was important
to keep the face-to-face element, hence full-
day workshops were adapted to be done over
a number of days, following the COVID-19
guidelines for face-to-face meetings.

Box 6: Action-oriented diagnostic for National Tariff Commission,
Pakistan (March – June 2021)
The main partner was the National Tariff Commission, the autonomous investigation
authority of Government of Pakistan. SDPI, SEDI’s representative in Pakistan engaged
with the chairperson of the Commission, who endorsed the exercised and appointed three
focal points who would join the design and planning phase and accompany SDPI in the
implementation.
NTC has a large leadership team, composed of the chairperson, the members and the
directors general. External stakeholders of NTC include some government departments
in addition to stakeholders from the private sector whose key input was obtained through
interviews (21) and focus groups discussions (2). The rest of the staff, composed of directors,
deputy directors, and assistant directors participated in the staggered workshops (5).

19
Key learning
points
Relationships with
government partners
Starting with trust
Repeatedly, and in line with the experiences
of previous evidence diagnostics shared in the
SEDI learning session, the teams found that
trusted, long-term relationships are key to
successful work with government agencies.
Such relationships are especially critical for
diagnostics like these, where a high degree of
trust is required for an agency to open its doors
to partners and to look frankly at what is and isn’t
working. In both Uganda and Pakistan, national
partners had strong pre-existing relationships

with the two partner agencies (the M&E
Directorate in the Office of the Prime Minister
and the NTC, respectively), which ensured
sufficient trust to undertake the diagnostics.
Setting expectations
Achieving a balance between SEDI’s and
government’s priorities was an important
means of sustaining initial trust and ensuring
that SEDI was grounded in engagement
with genuine government realities. The SEDI
programme’s holistic (usually lengthier)
approach did not always align easily with
the often-short-term interests of government
partners. At times, this made it challenging to
find the necessary common ground needed
to underpin a strong collaboration in which
partners could learn from each other. In some
cases, government agencies had immediate
needs and requests for specific areas of
technical input. Meanwhile, SEDI emphasised
a whole-picture view of how evidence is being
used, identifying barriers and opportunities so
as to direct energies to where there was most
potential for improvement. This was challenging
because while government partners sometimes
sought quick technical solution, SEDI was
proposing a much deeper, problem-driven
analysis first in order to identify sustainable and
effective interventions.
In some cases, SEDI was able to respond to
immediate government needs. For instance, in
Uganda SEDI embedded a technical advisor in
the M&E Directorate in the Office of the Prime
Minister to support the diagnostic process,
provide continuity after SEDI closure and offer
much-needed technical advisory support to the
Directorate on other aspects directly relevant
to SEDI’s objectives. While the SEDI team
embarked on this process with a systemic
approach to evidence, seeking to support
governments to address structural challenges
to improve evidence use, the team also learned
the importance of investing time in reaching
amicable trade-offs between short- and long-
term needs to lay foundations for partnership.
Emphasising ownership
To maximise the experience of government
partners, the SEDI team developed and
facilitated activities to elicit both implicit
and explicit knowledge. The diagnostics were
intended to give government partners a safe
space in which to explore their use of evidence,
which required them to own and be in control of
the process. The process followed the advice
of government partners in terms of who to
involve and how, when to carry out activities
and the appropriateness of tools. To help staff
in the Office of the Prime Minister and the NTC
to reflect on their own work, the SEDI team
emphasised the use of human-centred design
thinking, which aimed to tap into the collective
intelligence that already exists within teams.
While time and practical constraints meant the
SEDI team was not able to work in this way all
the time, they prioritised this spirit as much as
possible throughout the diagnostic process.
This approach was very different to what some
government agencies had come to expect from
international projects and was enabled in large
part due to the existing trust between partners
(as already described). In Pakistan, for example,
the NTC was more accustomed to working
with donors as providers of infrastructure or
services (including capacity building); the
SEDI programme’s approach, however, aimed
to co-design and co-implement a plan that
made sense for all parties involved. The team
invested additional time clarifying roles and
responsibilities with government partners. In
Uganda, the active involvement of the Office of
the Prime Minister M&E Directorate was critical
to establishing clear roles and responsibilities
for the Directorate itself but also for the SEDI
team and how they would complement one

21
another. This was documented in a terms of
reference and a workplan as part of a long and
iterative process, which helped to concretise
expectations and clarify the time investments
required of each team member.
Engendering a sense of ownership among
government partners was easier in Pakistan,
where the team was dealing with just one
institution. In Uganda, where Programme 18
involved more than one agency, this meant
building ownership of the diagnostic exercise
slowly, beginning with one agency (the M&E
Directorate in the Office of the Prime Minister).
From this, SEDI was able to progressively
grow ownership to the Programme Secretariat
under the Ministry of Finance, Planning and
Development,10
whose commitment has in
turn opened inroads to other agencies within
Programme 18.
Working with champions
A key element that underpinned and
reinforced ownership was the SEDI
programme’s active engagement of individual
champions within government partner
agencies. While this was a critical theme in
both countries, it took different forms for each
diagnostic. In Uganda, the M&E Directorate
within the Office of the Prime Minister has
itself acted as a champion, spearheading
the institutionalisation of evidence use within
the government over the past several years.
The team benefited from strong high-level
support from the M&E Directorate’s Assistant
Commissioner, who has been a champion of
EIDM for a number of years, having collaborated
with 3ie, Twende Mbele and UNDP among
others on several international capacity
development initiatives relating to evidence.
He was also nominated for an Africa Evidence
Network award in 2020.
In Pakistan, the champion was the NTC
chairperson, the authority who endorsed the
diagnostic exercise and helped to mobilise
people within the agency. However, the
experience in this case also illustrated the
importance of working with a small and
committed group of government staff, rather
than depending too heavily on the endorsement
of a single leader. Staff turnover increased with
the COVID-19 crisis and additional pressures
on the government. For example, during 2020,
the senior leadership at Ministry of Commerce
(a key stakeholder for the NTC) changed three
times. This required a degree of adaptability as
new leadership underwent orientation and to
factor in their own vision for the process. Having
buy-in from not only leadership but also a
team prevents having to start all over again
if the leader who initially started the process,
leaves their position. Overall, the combination
of high-level support and mid-level enthusiasm
helped the evidence agenda get traction in
Pakistan.

Methodological approach
Combining a research and a capacity
development approach
Combining a research approach with
a capacity development approach and
managing the two together – including
the tensions between them – necessitated
several internal conversations among
the SEDI design team and with government
agencies to clarify expectations. While a
diagnostic exercise has an important research
component, in the sense that it is an inquiry
process that follows specific parameters,
the SEDI diagnostics were also designed to
support capacity development. Through a co-
creative, participatory and inclusive process,
the diagnostic aimed to help all stakeholders
(government agencies and national partners)
to not only benefit from findings and insights,
but to better understand the role evidence
plays in their context, to develop their own skills
to conduct diagnostics in the future, and to
start to advance an evidence agenda in their
organisations.
Understanding the space for change:
authority, acceptance and ability
The AAA analysis undertaken as part of
the SEDI programme’s analysis phase
provided the diagnostic teams with important
indications of where there was space for
change or where effort would be needed to
build the enabling environment or agency’s
‘readiness’ to improve evidence use. In Pakistan,
for example, the authority (that is, the high-
level agency endorsement) to undertake the
diagnostic was clear but acceptance among
teams that would be affected by resulting the
change was less evident and needed further
exploration. The SEDI team therefore held
pre-diagnostic workshops to introduce the
diagnostic exercise and explain to government
partners what it meant and what would be
required of the agency’s team. In Uganda, the
authority to undertake the diagnostic was clear
among the M&E Directorate but less so within
the other core agencies that would be involved
in the process. To make sure that this was in
place, the M&E Directorate had to intervene and
bring other agencies up to speed. Ability (that
is, the practical aspects of starting a change
process) is always challenging, as it demands
time and resources, which are usually scarce.
In both the Pakistan and Uganda diagnostics,
the SEDI team needed to closely support the
government partner teams to make sure that
participation in the diagnostic wouldn’t take up
too much of their already limited time.
Going with the grain: linking up with
existing processes
Diagnostics in both countries identified
existing momentum with the agencies they
were working with. In Uganda, the transition from
sector to programme had been a window of
opportunity to take stock of what had gone well
and what hadn’t in terms of evidence use. From
the government’s perspective, the transition
was an opportunity to strengthen evidence use
as it was already discussing other evidence-
related efforts (e.g. a new M&E strategy for
the M&E Directorate and research agendas
for Programme 18 and others) and making
structural changes. By supporting a change
process that was already underway, the SEDI
programme could contribute an evidence lens to
the thinking about how to shape new structures
and processes. In Pakistan, the diagnostic
helped to generate the commitment necessary
to further the NTC’s ambition to strengthen its
role as a government think tank. In Uganda, the
diagnostic has spurred enthusiasm among the

23
M&E Directorate to develop an evidence-use
strategy for the programme area dedicated to
development plan implementation.
In practice, linking the diagnostic to existing
ongoing processes, especially when these
were in the early stages, was challenging
to navigate. For instance, conducting the
diagnostic during the transition from sector
to programme in Uganda revealed that not
all the agencies involved were at the same
pace of implementation nor had a common
understanding of the ongoing reforms and how
they were affecting or likely to affect evidence
use.
Prioritising focus areas
An important step was for partners to work
together to understand which areas of the
Context Matters Framework were most
relevant and to devote energy to those, rather
than comprehensively investigating the entire
tool. In Pakistan, as mentioned, a key priority
for the NTC was to improve its function as a
think tank, so external feedback was crucial;
this diagnostic therefore focused on the Context
Matters dimension of interrelationships within
and outside the public sector. In the case
of Uganda, the framework dimensions were
prioritised by addressing the main functions
of Programme 18: parties agreed to focus
on planning, monitoring and evaluation, and
budgeting.
The emphasis on government partner ownership
meant that the SEDI team had to negotiate
priorities and adapt expectations. For
instance, the SEDI team would have liked to
have developed a much deeper understanding
of gender equity and social inclusion dynamics
in the use of evidence in the government
partners. However, despite dedicating additional
time and resources to ensure these elements
were integrated throughout the diagnostic’s
planning and implementation (including
interviewing additional stakeholders), the SEDI
team found it difficult to achieve much depth
as it was not always a priority for the exercise
in some government agencies. In this context,
the team found this an effective compromise
between the SEDI project and government
partners.
Embracing politics to navigate
complex environments
The SEDI learning brief on the programme’s
analysis phase had noted the importance
of remaining continuously informed about
local political economy as a critical element
of implementation (Shaxson et al., 2021: 9).
The design of the diagnostic activities – such
as the inception meetings in the design stage,
the stakeholder mapping in Uganda and the
workshops – gave the SEDI team an ongoing
and evolving picture of who the key players
were and the power dynamics between them.
This enabled the SEDI team to adapt as new
relevant stakeholders emerged and dynamics
shifted. For instance, the Office of the President,
who was not initially a key stakeholder in
the diagnostic process, was integrated into
subsequent stages of the process in Uganda
following the release of the Programme
Implementation Plan, which made it responsible
for tracking NDP III results.
Shifting online: a radical change in
ways of working
The shift to an almost entirely virtual way
of working necessitated by the COVID-19
pandemic constituted a radical and unforeseen
change both for SEDI implementation partners,
and for government agencies in all the
programme’s countries of operation. The design
and implementation of the diagnostics had been
envisaged as largely face-to-face exercises,
involving in-person workshops and participatory

methodologies to build trust and relationships.
Working virtually presented many challenges:
from limited time and attention span to working
together in a hands-on way, difficulties in
having open and candid discussions, and
limited experience in using virtual tools for
online collaboration. The team learned that
online ways of working with governments
can require much more time to implement
and for changes to take effect than the in-
person approaches that both government and
practitioners had been accustomed to. However,
virtual working did facilitate the inclusion of
some stakeholders that would have otherwise
been unable to join because of their distance
from or travel time to the workshops. This was
relevant particularly in Uganda where several
agencies working under one programme would
converge via a hybrid of virtual and physical
meetings to deliberate on issues pertaining to
diagnostic and how it was going to be delivered.
Government partners appreciated some of
the new tools – including the use of Ideaflip
(an online, collaborative sticky note platform),
which they have adopted in their own virtual
engagements. The SEDI team had to learn by
doing and use online tools creatively to design
interactive group exercises as part of virtual
meetings.
Adapt framing of ‘evidence’ to the
context and partner
The diagnostic processes showed the need
to tailor language to the context. The Context
Matters Framework follows many in the
EIDM sector who have taken a broad view of
‘evidence’ in order to understand how it relates
to policy; however, the SEDI team found that
articulating what is meant by evidence
in more specific terms, according to the
context, helped to focus the diagnostic
exercise (INASP, 2016a; INASP, 2016b Shaxson
and Datta, 2016). In Pakistan, for example,
the team talked about ‘research’, ‘data’,
‘information’ and ‘statistics’, which made the
idea of ‘evidence’ more relatable to government
partners because these terms were part of
their daily work and tasks. Similarly, in Uganda,
‘evidence’ typically implied monitoring data and
statistics. This is a common interpretation. The
SEDI team therefore made a conscious effort
to encourage participants to be specific when
they talked about evidence – e.g. monitoring
data, evaluations, citizen feedback – and reflect
on the type of evidence they most commonly
use, what types of evidence might be missing
from this list, and the gender equity and social
inclusion implications embedded within different
types of evidence.
Management and team dynamics
Accompaniment is needed to support
a co-creative process
SEDI encouraged co-creation and adaptation
to local context but learned that this requires
significant time to do in a meaningful way. For
the diagnostics, the Context Matters Framework
was a means of operationalising this approach.
The Context Matters tool was new to country
leads EPRC and SDPI, and the SEDI team
therefore invested significant time in sharing
knowledge, experience and methodology, and
combining the expertise all partners brought
to the table. This process, and arriving at a
common understanding, took time – which

25
was even more drawn out given the pandemic
prevented the SEDI consortium from meeting
in person in order to co-construct a tailored
approach. Sessions to share knowledge,
experience and methodologies were carried
out online rather than during country visits by
international partners as had originally been
intended.
These challenges notwithstanding, the team
in Uganda was able to introduce and pilot
the co-creation approach through a series of
virtual workshops to co-design the delivery
framework, adapt it to a programme (versus a
single organisation) and build the capacity of
EPRC and M&E Directorate teams to conduct
the diagnostic. This process required a
substantial amount of iteration and a committed
and focused team that was able to rapidly
respond to windows of opportunity, prepare
inputs and make decisions about next steps.
In Pakistan, the team took a more condensed
approach, spending less time on the design of
the diagnostic in order to maximise a window of
opportunity for delivery of the diagnostic within
NTC.
Small and agile teams increase
adaptiveness
A clear learning point from SEDI’s experience
was the need to keep design teams small
and agile. Hands-on, iterative and adaptive
diagnostic exercises such as these require
teams to have a common focus and the space
to respond quickly to ever-changing political
environments. In Uganda, where the SEDI
programme first started working, the team
initially struggled to get multiple elements of
the SEDI programme involved in the design
and delivery of the diagnostic (MLA, TWP,
Communications, GESI). The team then found
that incorporating everyone’s point of view
was taking too long and risked the programme
becoming inefficient and unwieldy and losing
momentum. Over time, SEDI learned to be
smarter about when and how to seek inputs. For
example, the team identified key moments in the
diagnostic process where bringing feedback
and perspectives from outside the immediate
design team would be valuable (these were at
the end of inception phase, for feedback on the
delivery framework, and at the reporting phase,
for feedback on the first draft). In Pakistan
SEDI was able to keep the team small and
agile, inviting other SEDI experts for specific
input when required. Debrief and reflection
meetings after each of the major activities
(e.g. workshops) helped both country teams to
continually tailor their diagnostic approach.

Conclusion
The SEDI team’s experience of conducting
evidence diagnostics in Uganda and Pakistan
shows how the technical workstreams and the
country leads worked together to move from
‘thinking politically’ in the analysis phase of
the programme towards ‘working politically’ in
the implementation phase, using the authority,
acceptance and ability (AAA) model and the
Context Matters Framework. The reflections
shared in this brief emphasise the fundamental
importance of trusted relationships with the
key government partner agencies, which in
SEDI’s case pre-dated the programme. While
these relationships can be both expressed
and enabled by co-creative approaches, the
SEDI team found that this way of working was
unfamiliar for many involved and it was therefore
necessary to invest time in jointly discussing and
setting expectations for collaborative working
between the project team and its government
partners. To facilitate the agility required to work

27
politically and to inform intentional, strategic and
sustainable technical choices, SEDI teams also
made specific methodological decisions and
compromises.
27

Ahmed V. and Batool S. (2017) ‘India–Pakistan
Trade: Perspectives from the automobile sector
in Pakistan’. In: Taneja, N. and Dayal, I. (eds.)
India–Pakistan trade normalisation, Springer,
Singapore https://doi.org/10.1007/978-981-10-
2215-9_5
Andrews, M., Pritchett, L. and Woolcock, M.
(2017) ‘Building state capability: Evidence,
analysis, action’, Centre for Global Development,
Washington DC, https://bsc.cid.harvard.edu/
building-state-capability-evidence-analysis-
action
Daily Times (2020, July 10) ‘Awareness on
exports’ promotion facilities critical to tap global
opportunities’, https://dailytimes.com.pk/638457/
awareness-on-exports-promotion-facilities-
critical-to-tap-global-opportunities
Hayter, E. and Echt, L. (2018) ‘Context
Matters Framework Case Study: Supporting
organisational change to improve the use of
evidence in environmental policymaking in
Ghana’ https://www.inasp.info/publications/
context-matters-ghana
INASP (2016a) ‘Evidence-informed policy
making (EIPM) Toolkit’, Oxford, www.inasp.info/
publications/evidence-informed-policy-making-
eipm-toolkit
INASP (2016b) ‘Approaches to building capacity
for the use of evidence in policymaking’. https://
www.inasp.info/publications/approaches-
developing-capacity-use-evidence-policy-
making
Langer, L., Tripney, J. and Gough, D. (2016)
‘The science of using science: Researching the
use of research evidence in decision-making’,
EPPI-Centre, Social Science Research Unit,
UCL Institute of Education, University College
London.
Results for All (2017) ‘100+ government
mechanisms to advance the use of data and
evidence in policymaking: A landscape review’,
Washington DC, https://results4america.org/
our-work/results-for-all/results-global-landscape-
review
Shaxson, L. and Datta, A. (2016) ‘Guidelines and
good practices for evidence-informed policy-
making in a government department’, Overseas
Development Institute/South Africa Department
for Environmental Affairs, London, odi.org/en/
publications/guidelines-and-good-practices-
for-evidence-informed-policy-making-in-a-
government-department
Shaxson, L., Rocha Menocal, A., Leach, B.,
Hayter, E. and Harris, D. (2021) ‘Understanding
the demand and use of evidence through
a political economy+ approach: the SEDI
experience in Ghana, Pakistan and Uganda’,
SEDI learning brief 1, Strengthening Evidence
Use for Development Impact, Oxford.
Sustainable Development TV (2020, November
9) ‘How evidence on COVID-19 is being used
to support the trade sector?’, YouTube, www.
youtube.com/watch?v=JRrFYB18raM
Vogel, I. and Punton, M. (2018) ‘Final evaluation
of the Building Capacity to Use Research
Evidence programme’, Itad, www.itad.
com/wp-content/uploads/2020/02/BCURE-
FinalEvaluation-Report-25-Jan-2018-1.pdf
Weyrauch, V., Echt, L, and Suliman, S. (2016)
‘Knowledge into Policy: Going beyond Context
Matters’. Politics and Ideas and INASP, https://
www.inasp.info/contextmatters
References

29
Endnotes
1
For a more detailed explanation of the analysis phase
methodology see Shaxson et al. (2021)
2
The AAA model was developed by the Building State
Capability programme team at Harvard University
(see Andrews et al., 2017) and adapted for the SEDI
programme’s PEA (‘PEA+’). The AAA model is used for
analysing the space for change by looking at degrees
of authority, acceptance and ability within government
agencies.
3
For example, see Langer et al. (2016) and Results for All
(2017).
4
The toolbox was developed by SEDI’s technical workstream
leads and country leads and was grounded in SEDI’s theory
of change. While the tool was loosely based on INASP’s
internal learning and capacity development framework, the
levels of change and the interventions identified under each
are commonly used in the wider EIDM literature.
5
Politics & Ideas became Purpose & Ideas in 2021 (www.
purposeandideas.org).
6
The Context Matters Framework can be accessed via this
interactive webpage: https://www.inasp.info/contextmatters
7
Reproduced from: Weyrauch, V., Echt, L, and Suliman,
S. (2016) ‘Knowledge into Policy: Going beyond Context
Matters’. Politics and Ideas and INASP, https://www.inasp.
info/contextmatters
8
A learning session, which informed the diagnostic design
included speakers from government and external partners,
sharing experiences from two non-SEDI diagnostic
processes: the South Africa Department of Environmental
Affairs and Overseas Development Institute evidence
strategy process under the BCURE VakaYiko programme
from 2013–2017 (Shaxson and Datta, 2016); the INASP/
Politics & Ideas and Ghana Environmental Protection Agency
pilot of the Context Matters Framework in 2017–2018 (Hayter
and Echt, 2018); as well as emerging insights from SEDI’s
partnership between EPRC and Uganda Office of the Prime
Minister in 2020–2021.
9
In particular, the team made efforts to integrate gender,
equity and social inclusion within the questions being
explored by the diagnostics. These questions focused on
issues such as the representation of diverse groups in data
collection processes, evidence provision and review and
decision-making structures and leadership roles, and asked
about what elements of organisational culture promote
or hinder a more equitable working environment and
policymaking process.
10
For example, the Programme 18 Secretariat is based at
the Ministry of Finance, Planning and Development, where it
doubles as the secretariat the Private Sector Development
Programme. As part of the evidence diagnostic process,
representatives of the Secretariat attended a diagnostic
workshop and presented on their work. They were impressed
by the diagnostic on evidence use and the intention of
inculcating a culture of evidence use in Programme 18, and
subsequently extended an invitation to EPRC to deliver a
presentation on use of evidence for planning, budget and
M&E during a three-day workshop for planners of ministries,
departments and agencies under the Private Sector
Development and and Development Plan Implementation
(Programme 18) programmes. This workshop was attended
by 65 participants from over 25 ministries, departments and
agencies. The planners are key players in the evidence
ecosystem and being able to discuss issues of how to
embed evidence use in the two programmes with them was
a valuable learning opportunity for SEDI. These planners
also indicated that they looked forward to receiving the
recommendations from the evidence diagnostic.

About
SEDI
Strengthening Evidence Use for Development Impact (SEDI) is a five-year programme (2019-24)
that is working on increasing the use of evidence by policy makers in Uganda, Ghana, and Pakistan.
In partnership with country governments, this programme aims to develop capacity and promote
innovation in increasing evidence-informed decision making. SEDI is funded by UK’s Foreign,
Commonwealth & Development Office (FCDO).
The SEDI consortium is led by Oxford Policy Management and comprises national, international, and
regional partners. The national lead organisations – the African Center for Economic Transformation
in Ghana, the Economic Policy Research Centre in Uganda and the Sustainable Development
Policy Institute in Pakistan – provide programme leadership and coordination in each country.
These national organisations are authoritative voices in policy processes and will ensure effective
engagement and a sustainable legacy for SEDI.
The international partners – International Network for Advancing Science and Policy, the International
Initiative for Impact Evaluation, the Overseas Development Institute, and Oxford Policy Management
– as well as the regional partners – the African Institute for Development Policy and the Africa Centre
for Evidence – contribute their knowledge and years of experience in working with governments
across the world to promote evidence-informed development. They provide technical thought
partnership, facilitate cross-country learning, and collaborate on programme delivery.

31
Image Credits
Delegates listen on the first day of a workshop in
the Ugandan capital Kampala to draft Somalia’s
national gender policy. AMISOM
Businessmen holding pens and holding graph
paper are meeting to plan sales to meet targets
set in next year. Jirapong Manustrong
Increasing Productivity and Unleashing Growth
Day 1 May 26th 2012. IFPRI Pakistan Strategy
Support Program
Dorothy Okello (Coordinator, Women of Uganda
Network) and Imelda Anianbossou (CTA) Photos
from CTA’s workshop on “Making the next
generation ACP agriculture work for women”,
2018. CTA (Technical centre for agricultural and
rural cooperation
Women in Pakistan wait to vote. FCDO/ Rachel
Clayton
Financial businessman analyses the graph of
the company’s performance to create profits and
growth, market research reports and income
statistics, financial and accounting concept.
Davig Gyung
Man in black and white check shirt using
computer: Centers for Disease Control and
Prevention (CDC) activated its Emergency
Operations Center (EOC) to assist public health
partners in responding to the novel (new)
coronavirus outbreak first identified in Wuhan,
China. CDC
Woman in white and black stripped shirt writing
on white paper. Kojo Kwarteng
A discussion. Nadir Badshah, Sustainable
Development Policy Institute
First Cohort - YALI Accra Regional Leadership
Center. US Embassy Ghana
Panellist discussion. Nadir Badshah,
Sustainable Development Policy Institute

The content of this paper is the sole responsibility of the authors and does not
represent the opinions of SEDI or the Foreign, Commonwealth and Development
Office. Any errors and omissions are also the sole responsibility of the authors.
Please direct any comments or queries to sedi@opml.co.uk.
Suggested citation: Ahmed, V., Birabwa, E., Echt, L., Hayter, E., Menon, R., Perera,
A., Richards, C. (2021). ‘Using evidence diagnostics to think and work politically:
SEDI’s experience’. SEDI Learning Brief 5, Strengthening Evidence Use for
Development Impact, Oxford.

For more information
Email: sedi@opml.co.uk
Twitter: @SEDI_Programme

Evidence Diagnostic - Thinking & Working Politically

More Related Content

What's hot (20)

Similar to Evidence Diagnostic - Thinking & Working Politically (20)

More from Vaqar Ahmed (20)

Recently uploaded (20)

Evidence Diagnostic - Thinking & Working Politically