Results based management in development cooperation

REVIEW:
RESULTS BASED MANAGEMENT IN
DEVELOPMENT COOPERATION
Janet Vähämäki, Martin Schmidt, and Joakim Molander
30 November 2011
The review has been funded by Riksbankens Jubileumsfond. Arguments and opinions
expressed are those of the authors and do not necessarily reflect those of the sponsor.

2
Abstract
This
review
aims
to
briefly
introduce
and
analyse
current
literature
on
results-‐based
management
in
development
cooperation.
Our
review
includes
a
summary
and
analysis
of
selected
literature,
research,
blogs
and
presentations.
The
paper
provides
a
brief
historical
overview
of
where
and
when
results-‐based
management
was
introduced
to
development
cooperation,
some
lessons
and
experiences
that
can
be
drawn
from
evaluations
and
reviews
on
challenges
and
successes
in
its
implementation,
as
well
as
a
synopsis
of
the
global
initiatives
that
have
arisen
during
the
latest
years
in
support
of
or
as
a
response
to
the
agenda.
One
of
the
findings
of
the
review
is
that
the
basic
idea
behind
results-‐based
management
-‐
i.e.
that
we
need
performance
information
to
understand
and
adapt
to
development
contexts
and
to
make
informed
choices
–
seems
uncontested
by
actors
in
development
cooperation.
However,
the
management
practice,
per
se,
has
encountered
severe
challenges
and
difficulties
in
its
implementation,
and
is
being
questioned
by
practitioners
and
researchers.
Some
successes
are
noted
in
larger
partner-‐owned
programmes
and
individual
projects.
Different
results-‐based
management
approaches
have
been
tried
out
over
the
course
of
development
cooperation
history,
and
the
challenges
seem
to
have
been
repeated.
Some
of
the
initiatives
and
theories
that
have
arisen
as
a
response
to
the
agenda
provide
alternative
ways
and
methods
regarding
how
to
understand
complex
development
cooperation
contexts.

3
Contents
1.
Background................................................................................................................................4
2.
Method
and
selection
of
literature ..........................................................................................5
3.
Concepts
and
definitions...........................................................................................................7
4.
Short
historical
overview ........................................................................................................11
4.1.
Results
management
in
development
co-‐operation
in
the
1990s....................................13
4.2.
The
rationale
for
RBM
in
development
cooperation........................................................15
4.3
Global
Goal
setting
in
development
cooperation ............................................................16
5.
Experiences
from
evaluations
and
reviews ............................................................................18
5.1
Experiences
from
donor
institutional
level:.....................................................................20
5.2.
Experiences
from
project
level: .......................................................................................24
5.3
Experiences
from
partner
country
and
programme
level................................................26
6.
Some
theoretical
explanations ...............................................................................................30
6.1.
Complexity
theory ...........................................................................................................30
6.2.
Substantialist
versus
relational
thinking ..........................................................................30
6.3.
Social
change
theory........................................................................................................31
6.4.
Cultural
theory.................................................................................................................31
6.5.
Management
theory
difficult
–
but
there
is
no
other
option… .......................................32
6.6.
Some
conclusions ............................................................................................................33
7.
Current
international
initiatives
in
results
management ......................................................35
7.1.
Results
Based
Financing
–
Initiatives
that
focus
on
financial
incentives
for
the
achievement
of
results...................................................................................................35
7.2.
Initiatives
that
support
partner
countries .......................................................................39
7.3.
Initiatives
that
support
donors ........................................................................................40
7.4.
Global
measurement
initiatives.......................................................................................41
7.5.
Initiatives
questioning
the
agenda...................................................................................42
7.6.
Current
political
debate
and
The
Busan
Outcome
Document.........................................44
8.
Recommendations
and
lessons
provided
in
the
reports .......................................................46
9.
Conclusions .............................................................................................................................49
Annex
1.
Some
suggestions
for
further
research..............................................................................53
Literature: .........................................................................................................................................54

4
1. Background
The
discussion
about
what
methods
and
approaches
can
best
deliver
tangible
improvements
to
the
world’s
poorest
has
been
simmering
since
development
assistance
was
borne.
During
the
past
decade,
against
a
backdrop
of
growing
financial
constraints
and
a
tough
global
debate
on
the
efficacy
of
aid,
there
has
been
considerable
external
pressure
for
development
cooperation
agencies
to
reorient
their
management
systems
towards
effectiveness
and
results.
Thus,
results
management
systems
have
been
introduced
in
most
developed
country
government
sectors,
with
ongoing
implementation
in
most
developing
countries.
Fuelled
by
the
High
Level
Forum
on
Aid
Effectiveness
(HLF-‐4)
in
Busan,
South
Korea,
held
in
November-‐December
2011,
the
results
agenda
has
received
even
more
emphasis
and
is
currently
a
top
political
priority.
Renewed
scrutiny
has
intensified
calls
for
accountability
to
tax-‐payers
both
in
donor
and
recipient
countries,
and
the
need
for
results
information
to
improve
planning
and
analysis
of
what
works.
Also
important
to
the
discussion
is
that
without
demonstrable
results,
or
effective
provision
of
value
for
money,
there
is
a
risk
of
undermining
the
credibility
of
development
cooperation,
and
that
otherwise
willing
financiers
might
stay
away.
At
the
same
time,
the
results
agenda
has
encountered
a
more
structured
criticism.
Some
believe
that
the
agenda
has
narrowed
the
view
of
what
is
valued
and
how
value
is
measured;
and
that
it
is
counter-‐bureaucratically
promoting
an
“obsessive
measurement
disorder”1,
because
the
projects
that
are
most
transformational
are
also
the
least
measurable.
This
issue
has
been
debated
in
the
media,
in
various
influential
spheres
of
aid
and
politics,
and
has
manifested
itself
in
the
form
of
seminars
such
as:
“Can
Obsessive
Measurement
Disorder
be
avoided?”2
or
“A
results
take-‐over
of
aid
effectiveness?
-‐How
to
balance
multiple
or
competing
calls
for
more
accountability”3.
A
growing
amount
of
literature
and
research
points
to
the
difficulties
and
challenges
associated
with
implementing
the
results
agenda
within
the
development
community.
However,
there
exist
examples
of
successes,
as
well
as
several
recommendations
-‐
based
on
different
theories
-‐
on
how
to
improve
results
based
management
as
it
is
currently
practiced.
The
aim
of
this
paper
is
to
briefly
introduce
and
analyse
the
current
literature
on
results-‐based
management
in
development
cooperation.
It
has
been
financed
by
Riksbankens
Jubileumsfond
under
its
current
working
group
on
goals
and
results
in
public
administration
(MOR
gruppen).
The
group’s
central
theme
is
to
shed
light
on
how
targets
are
set
and
results
are
measured
in
public
affairs.
The
paper
will
focus
on
the
following
questions:
1)
Where
and
when
was
results-‐based
management
introduced
to
development
cooperation?
2)
Which
lessons/experiences
can
be
drawn
from
evaluations
and
reviews
on
challenges
and
successes
in
its
implementation?
3)
What
type
of
global
initiatives
exist
that
support
or
question
the
agenda?
4)
What
are
the
trends
and
countertrends
in
the
current
debate?
1
Stated
by
Andrew
Natsios
in
the
paper
“The
clash
of
counter-‐bureaucracy
and
development”
2
A
seminar
organised
by
the
Nordic
Africa
Institute,
FUF
and
Riksbankens
Jubileumsfond
in
May
2011
3
A
seminar
organised
by
ODI
–Overseas
Development
institute
in
July
2011

5
These
questions
provide
some
guidance
to
the
(discussed
in
the
conclusions)
question:
Is
it
possible
to
successfully
implement
the
results
agenda
this
time?
About the authors
This
report
has
been
written
by
individuals
who
bring
different
backgrounds
and
perspectives
to
the
topic.
We
consider
this
to
be
an
advantage.
Our
primary
ambition
has
been
to
present
an
optimally
nuanced
image
of
the
subject
that
embraces,
and
does
not
obscure,
a
variety
of
opinions
and
approaches.
Janet
Vähämäki
is
a
PhD-‐candidate
at
Uppsala
University’s
Department
of
Education.
She
obtained
previous
practical
experience
as
the
Acting
Head
of
the
Department
for
Methodologies
and
Effectiveness
at
Sida
(Swedish
International
Development
Cooperation
Agency),
Advisor
on
Aid
Effectiveness,
and
from
longstanding
experience
in
development
cooperation
both
in
the
field
(Honduras,
South
Africa
and
Brazil)
and
HQ.
Martin
Schmidt
is
CEO
of
SPM
Consultants
in
Stockholm
and
has
worked
with
results-‐based
management
for
Sida,
the
Swedish
Ministry
of
Foreign
Affairs,
and
international
development
agents
since
1998.
He
participated
in
several
“methods
development
projects”
(Sida,
UD,
the
Nordic
+
group)
and
worked
as
a
long-‐term
advisor
to
governments
on
planning
and
results-‐based
management
in
joint
sector
programmes,
including
PEDP-‐II
(Bangladesh),
PROAGRI
(Mozambique)
and
PRORURAL
(Nicaragua).
Joakim
Molander
is
Head
of
Development
Cooperation
at
the
Embassy
of
Sweden
to
Rwanda,
and
has
worked
with
issues
concerning
results-‐based
management
in
education
and
development
cooperation
since
1998
as
an
evaluator,
monitoring
and
evaluation
advisor,
evaluation
manager,
and
head
of
Sida’s
evaluation
department
from
2008-‐2011.
He
holds
a
PhD
in
philosophy,
and
has
taught
research
methodology
and
philosophy
of
science
at
Åbo
Academy
and
the
Mid
Sweden
University.
His
philosophy
of
science
work
has
been
published
in
Vetenskapsteoretiska
grunder.
Historia
och
begrepp.

6
2. Method and selection of literature
This
report
is
mainly
a
literature
review.
We
have
conducted
a
search
of
the
literature
for
different
chapter
purposes
(see
below),
have
used
literature
from
our
own
archives,
and
from
select
individuals
in
the
field.
Therefore
this
review
does
not
provide
a
comprehensive
perspective
of
all
literature
on
the
topic
–
there
is
certainly
information
and
literature
that
we
have
missed.
Except
from
literature
in
the
historical
part
of
the
study,
we
have
mainly
focused
on
literature
from
the
latest
decade,
i.e.
from
2000
and
further.
Chapter
4
–
Short
historical
overview:
In
the
historical
overview
we
have
relied
mainly
on
OECD/DAC
evaluation
and
aid
effectiveness
publications
from
the
1990s
and
the
beginning
of
the
2000s,
including
some
of
their
main
references.
Also,
management
theory
summaries
and
the
works
of
early
results
management
theorists
have
been
consulted.
Chapter
5
and
Chapter
6
-‐
Experiences
and
current
research.
The
reports
used
as
the
empirical
basis
for
this
chapter
have
been
found
by
using
the
search
words
“development
cooperation”
“aid”
“results
based
management”
on
Google
Scholar
and
SCOPUS,
on
OECD/DAC’s
webpage,
on
the
webpages
of
the
World
Bank,
IEG,
Centre
for
Global
Development
etc.
Research
reports
have
been
found
by
searching
in
Development
Journals,
e.g.:
Development
in
Practice,
Public
Administration
and
Development,
European
Journal
of
Development
Research,
International
journal
on
Development
Issues,
International
Human
Resource
Development
and
Management,
Journal
of
Management
Development,
Oxford
Development
Studies,
World
Development
and
the
IDS
Bulletin.
We
have
found
approximately
25
research
articles
dealing
with
results-‐based
management
and
aid,
around
7-‐8
evaluations/reviews
on
application
of
the
model
in
donor
agencies,
and
around
6-‐8
reports
dealing
with
the
application
of
RBM
in
partner
countries.
A
full
list
of
references
is
provided
at
the
end.
In
chapter
7
we
have
reviewed
current
blogs
and
discussion
forums
on
results-‐based
management.
We
have
also
followed
up
with
telephone
discussions
with
some
representatives
with
regard
to
varying
categories
for
different
global
initiatives4
4
“Initiatives
supporting
partner
countries”
–
Marjolaine
Nicod-‐
OECD/DAC;
Result
Based
Financing
-‐–
Rita
Perakis
from
the
Centre
for
Global
Development
and
in
charge
of
the
“Cash
on
Delivery-‐initiative”,
and
“Initiatives
questioning
the
whole
agenda”
–
Rosalind
Eyben/Institute
for
Development
Studies
and
founder
of
the
Big
Push
Forward.

7
3. Concepts and definitions
The
subject
of
review
in
this
paper
is
Results-‐based
management
(RBM)
as
it
is
currently
being
defined
in
development
cooperation
and
more
specifically
by
the
OECD/Development
Assistance
Committee
(DAC)
who
defines
RBM
as
“A
management
strategy
focusing
on
performance
and
achievement
of
outputs,
outcomes
and
impact”(OECD/DAC
Glossary
of
Key
Terms
in
Evaluation
and
Results
Based
Management,
2002).
The
definition
was
subsequently
expanded
in
a
working
paper
commissioned
by
the
Canadian
International
Development
Agency
–for
consideration
by
the
OECD/DAC.
The
following
definition
is
from
Meier
(2003):
“Results-‐Based
Management
(RBM)
is
a
management
strategy
aimed
at
achieving
important
changes
in
the
way
organisations
operate,
with
improving
performance
in
terms
of
results
as
the
central
orientation.
RBM
provides
the
management
framework
with
tools
for
strategic
planning,
risk
management,
performance
monitoring
and
evaluation.
It’s
primary
purpose
is
to
improve
efficiency
and
effectiveness
through
organisational
learning,
and
secondly
to
fulfil
accountability
obligations
through
performance
reporting.“
In
RBM
the
results
management
idea
is
conceptualised
by
way
of
an
image.
This
image
is
used
to
illustrate
a
mind-‐set
and
takes
the
shape
of
a
results
chain5.
The
main
idea
is
to
demonstrate
where
the
analytical
starting
point
is
–
on
the
outcome/impact
level
-‐
and
that
there
are
both
temporal
and
causal
dimensions
to
the
sequence
of
events
(chain)
that
lead
to
outcome
and
impact.
In
RBM,
the
idea
is
to
arrive
at
a
reasonable
results-‐chain
assumption,
i.e.
a
strategy
that
says
we
expect
X
to
happen
if
we
do
Y,
and
then
monitor
the
merit
of
that
assumption
in
implementation.
According
to
Meier
the
RBM
language
differs
from
precursors
such
as
Management
by
Objectives
and
Logical
Framework
Approach
(see
next
section
4),
where
significant
differences
lay
in
how
RBM
terms
(input,
output,
outcome
etc.)
are
defined
in
relationship
to
one
another.
5
Adapted
from
Meier
(2003)
p.
7.

In
2005
the
OECD/DAC
adopted
the
term
Management
for
Development
Results
(MfDR).
In
their
Sourcebook
on
MfDR
from
2006,
it
is
stated
that
results-‐based
management
is
nearly
synonymous
with
MfDR,
although
it
is
claimed
that
some
approaches
to
results-‐based
management
have
focused
only
on
accountability.
MfDR
is
defined
as
going
further
by
incorporating
newer
ideas
about
collaboration,
partnership,
country
ownership,
harmonisation,
and
alignment,
and
by
providing
a
higher
management
standard
by
asking
stakeholders
to
continuously
focus
on
country
outcome
performance,
rather
than
short-‐term
results.
In
the
Paris
Declaration
on
Aid
Effectiveness
(2005),
the
term
Managing
for
Results
is
used
to
denote
the
4th
(out
of
5)
principle
of
the
Declaration
under
which
“results-‐based
management”
and
“results-‐
oriented
reporting
and
assessment
frameworks”
are
highlighted
as
means
towards
achievement.
In
the
declaration,
managing
for
results
“means
managing
and
implementing
aid
in
a
way
that
focuses
on
the
desired
results
and
uses
information
to
improve
decision-‐making”.
In
clause
46,
partners
and
donors
jointly
commit
to
“work
together
in
a
participatory
approach
to
strengthen
country
capacities
and
demand
for
results
based
management”.
8
The
literature6
describing
RBM
refers
unanimously
to
the
importance
of
contextual
adaptability.
If
circumstances
or
performance
change,
you
should
be
ready
as
a
manager
(or
organisation)
to
respond
to
that
change,
and
alter
your
approach
if
necessary.
The
same
literature
refers
to
a
set
of
elements
that
enable
such
a
practice,
which
may
be
summarised
as:
1. An
analytic
and
performance-‐oriented
approach
to
understanding
what
to
do.
2. A
view
of
performance
that
is
centred
on
beneficiary
level
developments
(outcomes),
only
thereafter
moving
on
to
identify
what
resources,
activities,
and
immediate
tangible
results
(outputs)
are
necessary
to
get
there.
3. A
need
for
information
about
your
subject,
and
a
capacity
to
collect,
process
and
analyse
that
information
over
time.
4. A
mechanism
by
which
policy,
planning,
and
decision-‐making
can
be
influenced
by
performance
information.
Purposes of RBM
According
to
Ireland
(2003)
and
Binnedjikt
(2000),
RBM
serves
two
main
purposes:
management
improvement
(i.e.
learning,
improved
decision
making,
and
planning,
etc.)
and
performance
reporting
(accountability).
Some
also
use
RBM
for
the
purpose
of
performance-‐based
budgeting
–
i.e.
prioritising
budget
allocations
based
on
performance
information.
With
regard
to
development
cooperation,
these
purposes
are
valid
for
both
the
donor
organisation
and
the
partner
country.
Our
understanding
of
the
differing
purposes
is
illustrated
in
the
following
figure:
6
See
for
eg.
OECD/DAC
MfDR
Sourcebooks
2005,
2007
and
2008
for
summaries
and
references.
See
also
World
Bank.
(2004b).
Implementation
of
the
Agenda
on
Managing
for
Results:
Progress
Report
and
Annexes
(CODE2004-‐0017).
World
Bank,
Washington
DC.
Werner
Meier,
Discussion
paper
for
consideration
by
the
DAC
Working
Party
on
Aid
Effectiveness
and
Harmonization,
2003;
Results-‐Based
Management,
Towards
a
Common
Understanding
Among
Development
Cooperation
Agencies.
Moynihan,
D.
P.
(2005);
Goal-‐Based
Learning
and
the
Future
of
Performance
Management,
Public
Administration
Review
65(2):
pp.
203-‐16.
Mayne,
John;
Best
practices
in
results-‐based
management,
a
review
of
experience,
a
report
for
the
United
Nations
Secretariat,
July
2007.

The
graphic
serves
to
expand
on
the
previously
explained
purposes
of
RBM
with
an
external
and
internal
reporting/accountability/control
dimension,
because
many
organisations
differentiate
between
internal
and
external
results-‐management
practices.
A
differentiation
is
then
made
between
internal
results
management
within
the
agency;
such
as
performance
contracts
for
staff,
and
annual
planning,
budgeting
and
reporting
cycles
within
the
agency
and
performance
information
on
how
different
departments/units
have
performed,
etc.
External
results
management
is
characterised
by
the
results
that
are
attributable
to
the
performance
of
actions
supported
with
development
cooperation
funds,
and
with
accountability
and
demand
that
adhere
to
domestic
constituencies,
such
as
parliament
and
taxpayers.
It
is
clear
from
literature
on
the
subject
(see
further
evidence
on
this
in
chapter
4)
that
tensions
exist
between
the
different
purposes,
and
that
the
main
users
value
the
purposes
differently.
Domestic
reporting/accountability
for
citizens
in
donor
countries
is,
for
example,
highly
important
for
the
government
of
a
donor
country,
whereas
demonstrating
good
internal
control
is
of
greater
importance
for
the
responsible
agency.
Governments
in
donor
countries
often
justify
RBM
by
arguing
that
it
improves
accountability
and
learning/planning
in
partner
countries.
However,
based
on
the
evidence
from
the
literature
we
have
reviewed,
our
conclusion
is
that
demand
is
low
for
using
RBM
or
results
information
for
those
purposes
in
partner
countries,
and
similarly
for
the
purpose
of
learning
within
donor
countries.
One
explaining
variable
is
the
existence
of
unequal
power
relations,
often
portrayed
as
a
principal-‐agent
problem,
between
the
funder
and
the
recipient/s
paired
with
the
fact
that
aid
beneficiaries
have
limited
influence
of
how
results
are
defined
in
most
development
programmes.
The
funder
provides
support
and
demands
something
back
from
the
recipient
(e.g.
results/reporting),
and
one
frequent
critique
of
aid
is
that
the
accountability
of
partners
towards
the
donor
is
simply
stronger
than
their
accountability
to
their
domestic
citizens.
Moreover,
since
development
cooperation
funds
ultimately
come
from
taxpayers
in
donor
countries,
results
information
from
activities
funded
through
development
cooperation
is
mostly
used
for
accountability
to
domestic
audiences
in
donor
countries,
and
to
legitimise
donor
government
policies
in
regard
to
development
cooperation.
However,
accountability
mechanisms
and
RBM
practices
differ
somewhat
between
bilateral
donors
and
multilateral
donors/international
organisations.
Bilateral
donor
RBM
reform
is
normally
a
part
of
a
government’s
general
public
management
reform.
Hence
such
reforms
are
designed
for
domestic
policy
implementation,
rather
than
for
development
cooperation
in
third
countries.
This
creates
certain
challenges,
which
we
will
come
back
to
later.
However,
in
contrast
with
bilateral
donors,
multilateral
donors/international
organisations
are
owned
or
chaired
by
different
member
states,
9

10
and
normally
need
to
report
to
different
funders
on
results
of
their
specific
programmes.
Different
funders
often
require
incongruous
reporting
and
result
formats,
thereby,
perhaps,
making
results-‐
based
management
even
more
difficult
for
international
organisations.
However,
international
organisations
do
not
have
the
same
type
of
accountability
requirements
to
domestic
audiences
as
national
administrations
(i.e.
bilateral
donors)
do.
Does one RBM-model exist? Some limitations in this review
Much
of
the
critique
against
RBM
is
aimed
at
its
linear
thinking
and
the
technicalities
involved
in
complex
log
frames,
attribution,
aggregation
etc.
However,
it
is
clear
from
the
literature
that
organisations
apply
the
RBM
model
differently;
different
concepts
are
used
and
reforms
are
based
on
their
different
purposes
and
context.
What
is
important
depends
on
where
demand
comes
from.
Therefore:
A
singular
RBM
model
does
not
exist,
and
we
have
not
in
this
review
thoroughly
analysed
how
different
organisations
have
interpreted
RBM.
We
also
acknowledge
that
the
achievement
of
results
would
be
more
or
less
effective
with
different
results-‐based
management
strategies/models.
However,
we
have
not
analysed
if
certain
RBM
strategies
lead
to
comparatively
better
results,
or
are
more
effective
per
se
–
this
would
simply
be
a
much
larger
task
than
we
set
out
for
this
review.
We
have
analysed
the
implications
of
the
introduction
of
the
management
perspective
as
such,
as
referred
to
in
the
literature
and
how
evaluators/researchers
have
concluded
that
RBM
has
been
applied
in
the
organisation.
Logically
this
is
a
limitation,
because
evaluation
responses
could
vary
depending
on
the
purposes
used
during
their
introduction.
However,
as
stated
in
chapter
4,
we
have
found
that
the
evaluations/reviews
express
quite
similar
experiences,
and
have
thus
-‐
in
this
review
-‐
chosen
to
categorise
the
different
experiences.
To
make
clear
distinctions:
in
this
review
we
will
refer
to
the
term
results-‐based
management
(RBM)
as
the
management
perspective
that
was
developed
during
the
1990s,
and
as
defined
by
DAC
(2002)
and
Meier
2003
(see
above).
With
the
term
results
management,
we
will
refer
to
the
much
older
general
management
idea
of
results-‐oriented
management
behaviour.
This
general
idea
will
be
outlined
in
the
next
chapter
(4).
We
will
also
refer
to
the
results
agenda,
by
which
we
mean
the
specific
drive
in
development
cooperation
that,
through
various
policies
and
models,
advocate
and
encourage
the
use
of
results-‐oriented
management
practices
in
aid
delivery.

11
4. Short historical overview
Tracing
results
management
to
a
specific
time
or
setting
is
not
easily
done.
It
has
been
a
part
of
Management
Theory
in
a
broad
sense
throughout
the
20th
century.
However,
the
ideas
and
practices
that
are
today
associated
with
results-‐based
management
and
related
concepts
in
development
aid
probably
began
to
form
just
around
the
Second
World
War.
In
the
landmark
paper
The
Practice
of
Management
from
1954,
Peter
Drucker
suggests
that
successful
management
is
strongly
associated
with
executives
thinking
about
performance
in
a
particular
way.
He
notes
that
when
managers
analyse
a
situation
and
act
from
the
point
of
view
of
performance,
objectives,
and
results,
they
are
significantly
more
successful
than
when
they
do
so
from
the
point
of
view
of
budgets
and
operation
programmes.
Drucker
emphasises
the
learning
aspect
of
this
way
of
thinking,
and
discourages
the
application
of
deterministic
or
mechanical
models
to
results
management
seen
in
other
areas
of
management
theory.
Drucker
emphasises
that
his
perspective
is
something
that
takes
place
foremost
in
the
mind.
When
adopted
by
the
manager,
it
has
consequences
for
the
way
in
which
he
or
she
organises
working
processes.
In
1964,
Drucker
developed
his
perspective
in
the
book
Managing
for
Results.
It
focuses
on
strategic
choice
and
the
thought
processes
that
use
evidence
and
performance
for
making
informed
decisions.
With
this
mind-‐set,
managers
engage
in
and
encourage
results-‐analyses
and
an
open-‐minded
approach
where
the
main
concern
is
adaptation
in
view
of
changing
circumstances
and
performance,
rather
than
control
and
predictability.
In
The
Practice
of
Management
from
1954,
Drucker
had
popularised
the
methodological
concept
of
Management
by
Objectives
(MBO).
In
the
1960s
and
70s,
MBO
emerged
and
thrived
in
public
administrations
as
well
as
in
private
businesses.
MBO
can
best
be
described
as
a
participatory
working
tool
designed
to
focus
the
mind
on
what’s
important
(objectives
and
performance).
In
fact,
the
tool
more
or
less
assumes
that
the
pervading
management
perspective
is
not
results
oriented,
but
that
it
should
become
so
through
the
use
of
MBO.
Rival
or
alternative
systems
of
orienting
management
towards
results
developed
soon
thereafter.
In
1969
USAID
asked
Leon
J.
Rosenburg
of
Fry
Consultants
to
develop
a
model
for
development
aid
project
management.
The
result
was
the
Logical
Framework
Approach
(LFA)
which
swiftly
came
into
fashion.
It
spread
unevenly
from
development
aid
to
public
administrations
in
OECD
countries
during
the
1970s
and
80s,
and
in
the
process
was
rebranded
under
new
acronyms
as
it
was
adopted
by
new
practitioners
-‐
Goal
Oriented
Project
Planning
(GOPP)
and
Objectives
Oriented
Project
Planning
(OOPP)
are
both
close
variants
of
the
LFA.
During
the
late
1980s
and
early
1990s
many
OECD
countries
undertook
extensive
public
sector
reforms
in
response
to
economic,
social,
and
political
pressures.
This
process
included
a
strong
emphasis
on
results
management.
However,
there
was
a
simultaneous
realisation
that
results-‐
management
was
not
the
prevailing
management
perspective
in
public
administrations.
By
contrast,

12
public
management
-‐
and
development
aid
management
-‐
had
been
characterised
by
their
orientation
towards
budgeting,
activity,
and
control
(DAC
2000).
In
the
late
1960s,
the
Planning,
Programming
and
Budgeting
Systems
(PPBS)
approach
put
an
emphasis
on
financial
planning
and
cost
accounting.
In
this
perspective,
the
management
of
inputs,
i.e.
human
resources
and
operating
and
capital
costs,
was
of
paramount
importance
in
demonstrating
management
control
over
the
allocation
and
use
of
financial
resources.
Both
in
public
administrations
and
in
development
aid,
this
tradition
promoted
the
Programme
Management
By
Activity
(PMBA)
approach
during
the
1970s
and
80s,
combining
several
tools
and
techniques
to
plan
and
schedule
activities
(including
e.g.
the
Work
Breakdown
Structure,
WBS,
the
Gantt
Chart,
the
Critical
Path
Method,
CPM,
and
the
Programme
Evaluation
and
Review
Technique,
PERT).
These
approaches
and
techniques
emphasised
the
implementation
of
activities
according
to
a
planned
schedule,
and
were
derived
from
the
fields
of
construction
engineering
and
systems
management.
In
a
sense,
they
represented
an
alternative
to
Drucker’s
theories
of
the
1940s
and
50s
regarding
the
character
of
successful
management.
In
a
parallel
development,
the
1970s,
80s,
and
90s
saw
a
number
of
approaches
that
combined
results
management
with
financial
management.
In
the
business
sphere,
the
Corporate
Performance
Management
(CPM)
approach
brought
together
budgets
and
results-‐orientation
with
a
view
to
associate
costs
with
achieved
results.
In
the
1980s
and
90s
the
Performance-‐based
budgeting
(PBB)
approach
surfaced
in
public
administration
and
development
aid
with
a
similar
view
to
budgeting
and
performance
management.
Although
powerfully
influential,
these
approaches
had
difficulty
achieving
widespread
recognition
in
public
management.
In
1997
the
OECD
published
a
report
-‐
In
Search
of
Results:
Public
Management
Practices
-‐
that
examined
the
public
reform
efforts
of
the
past
decade
in
ten
OECD
countries.
It
concluded
that
variants
of
results
management
had
been
a
prominent,
even
dominant,
feature
in
all
ten.
This
is
also
the
period
in
which
results
management
became
a
basic
component
of
modern
public
management
practices
(Meier
2003).
In
Canada,
one
of
the
ten
countries
of
the
OECD
study,
the
Auditor
General
and
the
Treasury
Board
Secretariat
had
been
the
primary
promoters
of
public
reform
and
performance
management.
During
the
early
1990s
this
is
probably
where
the
term
Results-‐Based
Management
(RBM)
was
coined.
During
the
1990s
it
was
introduced
incrementally
in
several
bilateral
and
multilateral
agencies
in
development
cooperation
(DAC
2000,
p.
137).
Here,
RBM
represents
a
return
to
the
notions
of
Drucker,
and
his
insistence
on
successful
results
orientation
as
a
mind-‐set
and
a
perspective
on
management,
rather
than
a
precise
set
of
instructions.
RBM
can
also
be
viewed
in
the
context
of
the
New
Public
Management
(NPM)
philosophy
introduced
in
the
1980s
which
was
intended
to
modernise
public
management
by
making
it
more
market
oriented.
Several
terms
have
been
used
in
NPM
to
denote
a
results
management
approach.
In
development
aid,
RBM
became
the
preferred
demarcation,
and
in
the
first
years
of
the
new
7
Binnendijkt
notes
five
bilateral
and
two
multilateral
agencies
with
considerable
experience
in
establishing
results-‐based
management
systems;
USAID,
AusAID,
DANIDA,
DFID,
CIDA,
World
Bank,
and
the
UNDP.

13
millennium
it
was
adopted
by
the
OECD/DAC
and
subsequently
by
the
development
community
at
large.
With
MBO,
LFA
and
their
variations,
the
results
management
idea
had
been
applied
in
practical
methods,
step-‐by-‐step
models
and
tool
kits
-‐
in
LFA,
the
original
Rosenburg
working
tool
is
a
26
field
results-‐matrix
summarising
project
logic.
By
contrast,
in
RBM,
focus
is
diverted
away
from
individual
operations
to
the
working
and
planning
processes
of
an
organisation
and
it’s
environment,
thus
proposing
a
results-‐oriented
perspective
at
the
heart
of
organisational
thinking
and
practice.
This
is
not
to
say,
however,
that
RBM
implementation
in
the
1990s
was
adopted
as
it
was
intended.
In
fact,
a
series
of
studies
from
the
time
of
the
OECD
1997
study
on
public
reforms
concluded
that
a
more
holistic
RBM
approach
was
truly
challenging
because
of
its
organisation-‐wide
ambition
and
coverage
(a
summary
and
extensive
review
of
this
literature
is
found
in
Mayne
2007).
An
alternative
approach
to
analytic
and
outcome
oriented
thinking,
outcome
mapping,
was
introduced
in
2001
by
the
IDRC
(International
Development
Research
Centre).
Outcome
mapping
builds
on
Barry
Kibel’s
Outcome
Engineering
approach
and
aims
at
shifting
the
focus
of
results
management
both
from
organisational
performance
(outputs)
and
impact
(seen
as
significant
and
large
scale
changes
in
society
caused
by
development
interventions).
Instead
it
focuses
on
outcomes
defined
as
changes
in
the
behaviour,
relationships
and
actions
of
the
people,
groups,
and
organizations
with
whom
a
programme
works
directly.
The
idea
is
that
a
management
approach
which
is
aimed
at
helping
programme
learning
and
improvement
(as
is
the
core
of
RBM),
needs
to
be
focused
on
results
which
can
be
attributed
to
the
programme
itself.
According
to
Sarah
Earl,
Fred
Carden
and
Terry
Smutylo
who
developed
the
method,
this
contrasts
with
the
traditional
approach
in
development
cooperation,
where
donors
expect
accountability
for
impacts
over
which
no
single
agency
has
control
or
can
realistically
claim
full
credit.
(Earl,
Carden
Smutylo
2001)
4.1. Results management in development cooperation in the 1990s
The
results
agenda
in
development
received
renewed
emphasis
in
the
1990s.
As
part
of
a
global
discussion
on
the
most
effective
strategies
for
financing
development
interventions,
studies
on
progress
in
developing
countries
continuously
indicated
that
a
majority
of
development
aid
had,
at
best,
marginal
and
short
lived
effects.
The
international
response
to
these
experiences
was
to
challenge
the
“project
mode”
of
cooperation.
Throughout
the
history
of
development
cooperation,
a
vast
expansion
has
taken
place
in
a
number
of
institutions
and
actors
in
developing
countries.
In
part,
this
expansion
can
be
attributed
to
“project-‐
mode”
co-‐operation,
which
exponentially
increases
the
availability
of
short-‐term
financing
via
a
series
of
small-‐scale,
unrelated,
and
uncoordinated
contributions.
By
the
early
1990s,
the
situation
was
at
such
a
pitch
that
almost
all
evaluations
of
development
projects
concluded
that
they
seldom
achieved
targets
beyond
their
own
internal
life.
Almost
by
default,
country
and
sector-‐wide
studies
uncovered
duplications,
(vast)
omissions,
conflicting
practices,
and
unsustainable
results
wherever
they
turned
their
attention.8
8
In
a
study
by
Forrester
and
McLoughlin,
Sector
Wide
Approaches
and
Financial
Accountability,
Dublin
1998,
the
authors
convincingly
summarise
the
adverse
consequences
of
project
support.
The
message
from
the
collected
experience
of
the
development
community
was
that
no
matter
how
successful
the
individual
project,
the
majority
failed
to
leave
traces

Thus
was
it
concluded
that
fragmentation
in
project
development
cooperation
made
it
difficult
for
national
public
reforms
to
be
pursued
in
a
sensible
way.
Policy
makers
could
not
achieve
the
overview
necessary
to
make
sound
priorities
with
scarce
resources.
A
multitude
of
projects,
approaches
and
actors
obscured
strategic
choice.
As
a
result,
existing
records
of
sector
performance
were
mostly
fragmented
or
non-‐existent
(EC
2002).
The
immediate
effect
was
that
programme
support
came
into
swing.
Large
investment
programmes
had
already
developed
by
the
1980s.
Particularly,
the
development
banks
had
introduced
larger
and
more
comprehensive
“structural
adjustment
programmes”,
i.e.
programmes
with
the
goal
of
reducing
fiscal
imbalances
in
developing
countries.
However,
these
were
still
overwhelmingly
bilateral
affairs,
involving
a
donor
agency/bank
and
a
recipient
government,
and
thereby
taking
the
shape
of
very
large
projects.
What
mainly
set
them
apart
from
the
next
generation
of
“programmes”
that
was
to
follow
in
the
1990s,
was
their
often
limited
sector
coverage
and
low
levels
of
domestic
ownership/stewardship.
In
the
1990s,
programmes
were
to
be
based
on
a
national
strategy,
and
attempted
to
summon
all
major
donors
in
a
sector
under
the
same
framework.
The
programme
definition
essentially
became
what
appeared
in
the
Paris
Declaration
on
Aid
Effectiveness
more
than
a
decade
later9.
During
this
transformation
process,
more
specifically,
experiences
began
to
assemble
that
pointed
to
the
need
to
push
the
results
management
agenda
a
step
further.
Two
broad
strands
of
development
can
be
identified.
The
first
is
that
the
existing
project
results
management
idea
was
widely
criticised.
Results
frameworks,
often
manifested
in
an
LFA
results
matrix,
were
seldom
if
ever
put
to
use
in
day-‐to-‐day
project
management.
Experience
instead
indicated
that
the
matrices
were
filled
out
at
the
beginning
of
a
project,
only
to
be
brought
back
years
later
when
it
was
time
for
an
evaluation10.
This
meant
that
results
management
was
limited.
Reports
indicated
that
results-‐oriented
tools
introduced
an
initial
focus
on
objectives
(in
the
sense
that
they
were
justified).
However,
during
implementation,
projects
remained
inflexible
and
unresponsive
to
outcome
developments
because
they
were
not
monitored
until
it
was
too
late,
if
at
all.
It
was
also
concluded
that
tools
such
as
the
LFA
were
considered
by
its
practitioners
as
add-‐ons
to
already
existing
management
practices.
Critics
also
pointed
to
the
fact
that
the
LFA
model
was
itself
never
evaluated,
that
the
model
created
mistrust
within
organisations,
and
that
causal
thinking
required
by
the
model
introduced
difficulties
in
project
management.
(Des
Gasper,
2000
and
Eyben)
The
second
is
that
when
joint
programmes
began
bringing
several
donors
together
over
joint
strategies,
the
inter-‐agency
dialogue
revealed
wild
differences
in
results
perspectives,
terminology,
typologies,
and
the
general
meaning
of
results
management
and
associated
objectives
and
indicators.
(EU
2002)
This
happened
simultaneously
with
the
realisation
of
the
growing
need
for
solid
information
to
guide
programme
priorities
and
strategic
choice.
With
a
stroke,
the
“national”
or
“sector
programme”,
because
of
its
coverage,
had
brought
development
aid
affairs
up
alongside
beyond
their
own
confines.
As
a
group,
with
individual
exceptions
of
course,
they
failed
to
impact
positively
on
sector
or
national
development.
This
was
a
truly
disappointing
finding.
9
See
Appendix
A
to
the
Paris
Declaration
on
Aid
Effectiveness,
Methodological
notes
of
the
indicators
of
progress,
note
on
indicator
9.
10
This
practice
is
part
of
what
is
sometimes
referred
to
as
an
ex-‐post
evaluation
tradition
in
development
aid;
in
contrast
with
a
results
oriented
management
perspective.
In
this
tradition
evaluations
are
conducted
when
operations
are
concluded,
often
as
a
way
of
identifying
lessons
learned
and
formulating
best
practices.
In
a
results
oriented
perspective,
monitoring
and
evaluation
is
part
of
implementation.
14

15
national
policy-‐making.
This
had
only
seldom
been
the
case
in
project
mode
cooperation.
The
need
for
quality
information
-‐
and
reliable
time-‐series
of
it
-‐
was
brought
to
a
different
level.
When
agency
representatives
collaborated
during
the
1990s,
the
diaspora
of
results
management
tools
from
varying
development
agencies,
including
the
MBO
and
LFA,
were
being
brought
back
together
again.
As
a
result,
the
many
manifestations
of
tools
were
put
on
display,
resulting
in
a
deep
and
lingering
confusion.
(EC
2002
Guidelines)
The
confusion
was
not
only
due
to
different
results
perspectives
and
tools;
it
was
also
a
consequence
of
a
mounting
organisational
resistance
to
results
management.
Alternative
management
perspectives,
prescriptive
and
process
oriented,
were
still
very
much
at
the
heart
of
organisational
thinking.
These
findings
indicated
that
the
results
perspective,
which
suggested
adaptation
to
local
conditions
and
an
analytical
and
critical
approach,
was
challenging
the
foundations
of
Western
development
aid
policies
which
were,
and
still
are,
very
much
centred
on
“their
own”
policy
objectives.
(DAC
peer
reviews
1998-‐2007,
Hervio
et
al
2000)
4.2. The rationale for RBM in development cooperation
Towards
the
end
of
the
1990s,
and
beginning
of
the
2000s,
the
RBM
perspective
was
formed
against
a
backdrop
of
lessons
including
difficulties
of
project
aid,
low
levels
of
ownership,
and
unclear
development
results.
A
new
rationale
for
results
orientation
in
development
aid
emerged
that
took
on
a
different
shape
from
that
which
had
motivated
LFA
and
its
siblings
three
decades
earlier
-‐
basically
intent
of
sharpening
project
logic
and
results
achievement.
Several
studies
pointed
towards
the
need
to
deal
with
what
were
seen
as
structural
problems
in
development
cooperation.
And
results-‐oriented
practices,
or
the
lack
of
them,
came
to
the
centre
of
attention.
In
the
year
2000,
a
broad
in-‐country
evaluation
under
Hervio
(Hervio
et.
al.
2000)
drew
some
notable
conclusions
about
the
non
results-‐oriented
practices
in
programme
aid
during
the
previous
decade:
a. Reform
programmes
had
been
pursued
with
little
regard
for
local
context
and
with
an
overwhelming
stress
on
a
conditionality
of
measures
–
thus
lowering
ownership
and
obscuring
reform
content
and
rationale
for
national
decision-‐makers.
b. Programmes
essentially
had
been
run
without
continuous
follow-‐up
mechanisms
to
trace
sector
outcome
developments
over
time
–
meaning
that
key
policy
decisions
as
well
as
key
programme
priorities
had
been
set
without
proper
knowledge
of
sector
or
programme
performance.
The
study
created
a
stir
for
its
conclusions
and
recommendations
for
a
shift
in
mind-‐set.
It
suggested
the
need
to:
• Conduct
a
constructive
reform
dialogue
with
partners
from
the
point
of
view
of
sector
performance,
rather
than
conditionality,
budget
execution,
or
activities.
• Make
overall
strategic
choices
based
to
a
larger
extent
on
country
development
patterns.
• Make
sound
programme
priorities
based
on
analyses
of
country
and
programme
outcomes.

A
series
of
other
studies
came
to
similar
conclusions
during
this
period
(for
a
guide
and
summary
see
White
and
Dijkstra
2003).
The
rationale
for
a
renewed
and
reoriented
focus
on
results
that
emerged
included11:
16
1.
Increasing
contextualisation
–
national
development
patterns
(outcome/impact)
should
guide
strategy
–
as
a
means
to
stimulate
a
capacity
for
national
policy
making;
2.
To
regularly
monitor
national,
sector
and
programme
performance
on
outcome
level
so
that
priorities
could
adapt
over
time
in
response
to
changing
conditions,
and
in
the
process
strengthen
national
performance
assessment
frameworks
to
support
future
policy
and
decision-‐making;
3.
Promoting
a
dialogue
based
on
performance
rather
than
prescriptions
or
budget
and
activity
programmes
to
promote
ownership
and
joint
understanding.
Summarising
this
far,
in
the
historical
overview
of
results
management
practices
in
development
aid
recurrent
problems
with
application
stand
out.
In
most
settings,
it
seems
the
idea
is
at
odds
or
crowding
with
other
management
ideas
and
practices.
Much
reported
is
a
tendency
to
translate
the
results
management
idea
into
formal
or
mechanistic
models
in
contrast
with
analytic,
adaptable
and
learning
oriented
thinking.
Other
regular
problems
include
finding
the
right
level
of
analysis,
the
right
indicators,
and
troubles
with
attribution.
Against
such
observations
it
is
often
deduced
that
results
management
has
not
been
put
to
use
as
prescribed,
which
may
also
serve
to
conclude
this
chapter.
4.3 Global goal setting in development cooperation
During
the
course
of
development
cooperation
history,
some
attempts
have
been
made
to
set
global
goals12.
Here
we
will
mention
some
of
the
goals
that
have
been
perhaps
the
most
influential
in
development
cooperation:
1)
The
0,7%
disbursement
goal
–the
HOW
MUCH
–goal
2)The
Millennium
Declaration,
from
2000,
with
its
goals
on
WHAT
to
do
and
3)The
Paris
Declaration,
from
2005,
with
its
goals
and
principles
on
HOW
to
do
it.
By
the
1960s
strenuous
discussions
were
taking
place
on
the
size
and
volume
of
aid
required
to
result
in
poverty
reduction
in
the
world.
A
UN
conference
in
1973
determined
that
the
percentage
of
a
country’s
GDP
to
be
utilised
for
development
cooperation
should
be
0.7%13.
Since
then
the
disbursement
goal
has
been
hotly
debated
–
with
much
pressure
and
competition
regarding
disbursement
totals
at
project,
programme,
and
global
levels,
and
additional
funding
required
to
meet
the
goals.
Also
within
aid
administrations,
disbursements
are
continuously
monitored.14
This
approach
has
encountered
a
lot
of
criticism,
for
example
by
Moss
(2011)
who
argues
that
a
goal
11
See
the
Memorandums
of
the
High
Level
Roundtables
on
Development
Results
in
Washington
(2002)
and
Marrakech
(2004),
and
the
OECD/DAC
working
party
on
aid
effectiveness;
Managing
for
development
results,
principles
in
action,
April
2005.
12
Already
in
the
70s
efforts
were
made
to
agree
on
a
joint
goal
to
reduce
poverty
reduction
(see
for
eg.
Odén
2006).
However,
through
the
MDGs
a
joint
agreement
on
development
goals
in
various
areas
were
made.
13
However,
in
a
research
paper
by
Moss
(2007)
the
origins
of
the
0,7
%
disbursement
goal
are
discussed.
The
paper
shows
that
it
arose
from
an
economic
model
with
no
modern
credibility,
and
that
–
contrary
to
conventional
wisdom
–
none
of
the
UN
documents
contains
a
promise
to
meet
the
goal.
They
state
that
they
have
actually
not
found
a
target
has
actually
no
was
mostly
arbitrary,
based
on
a
series
of
assumptions
that
no
longer
are
true,
and
justified
by
a
model
that
is
no
longer
considered
credible.
14
For
actors
other
than
nations,
such
as
NGOs
or
IGOs,
the
disbursement
goal
has
equivalents
in
the
need
to
attract
and
spend
as
much
as
possible
to
safeguard
organisations
and
to
lobby/advocate
particular
objectives
before
others.

divorced
from
political
realities
is
counterproductive
as
a
lobbying
tool,
and
that
assessing
financing
needs
in
developing
countries
is
extremely
difficult.
A
financing
goal
tells
us
nothing
about
the
realities
in
developing
countries.
This
view
seems
to
be
commonly
supported
by
practitioners
in
development
cooperation.
However,
despite
this,
the
disbursement
goal
has
perhaps
always
been
the
most
practical
and
powerful
goal
in
development
cooperation.
17
Goals
oriented
to
WHAT
to
do
arose
with
the
UN
Millennium
Declaration
(2000),
and
with
it
the
Millennium
Development
Goals,
MDG.
The
MDGs
formulate
eight
goals
and
21
targets
to
be
achieved
by
2015
and
provided
for
the
first
time
in
development
cooperation
history
common
results
to
be
achieved
on
a
global
level.
With
the
Millennium
Declaration,
the
OECD/DAC
embarked
on
a
course
that
was
to
culminate
in
2005
in
the
Paris
Declaration
on
Aid
Effectiveness,
which
was
to
demonstrate
a
global
understanding
on
HOW
to
achieve
the
MDGs.
In
the
process,
the
results
management
perspective
became
an
integral
part
of
global
development
aid
policy.
Out
of
the
five
principles
of
the
Paris
Declaration,
two
points
directly
to
results
management
practices
(Managing
for
Results
and
Mutual
Accountability).
Both
partner
countries
and
donors
committed
in
the
Declaration
“to
strengthening
results-‐based
management
in
development
cooperation”.
The
principle
of
ownership
was
highlighted
as
a
prerequisite
for
the
other
principles
in
the
Declaration.
A
series
of
international
agreements
and
declarations
preceded
and
followed
the
Paris
Declaration.15
In
part,
these
repeated
affirmations
of
the
principles
of
the
Declaration
-‐
ownership,
harmonisation,
alignment,
managing
for
results,
and
mutual
accountability
-‐
also
confirm
a
struggle
to
enforce
a
new
development
agenda.
Some
development
partners
opposed
the
agenda
with
arguments
that
it
was
an
attempt
by
countries
in
the
North
to
impose
their
conditions
on
countries
in
the
South.
Within
donor
administrations,
the
agenda
met
criticism
for
trying
to
impinge
upon
the
complex
relations
in
development
cooperation.
Obviously,
the
agenda
proposed
changes
to
the
methods
of
managing
development
cooperation,
and
the
call
for
reinforced
results
orientation
was
a
prominent
element
alongside
the
call
for
programme-‐based
approaches.
15
Monterrey
Consensus
(2002);
Washington:
First
Roundtable
on
Development
Results
(2002);
Rome
Declaration
(2003);
Marrakech:
Second
Roundtable
on
Development
Results
(2004);
Paris
Declaration
on
Aid
Effectiveness
(2005);
Hanoi:
Third
Roundtable
on
Managing
for
Development
Results
(2007);
Accra:
Third
High
Level
Forum
on
Aid
Effectiveness
(2008);
Busan:
High
Level
Forum
on
Aid
Effectiveness
(2011).

18
5. Experiences from evaluations and reviews
In
this
chapter
we
will
provide
an
overview
of
challenges
as
well
as
lessons
from
the
introduction
of
results-‐based
management
in
development
cooperation.
We
will
use
three
organisational
levels
to
illustrate
the
experiences.
The
first
level
-‐
the
establishment
of
performance
measurement
and
management
systems
at
the
corporate
or
agency-‐wide
level
-‐
is
where
we
have
found
and
reviewed
the
most
experiences
and
evaluations.
The
second
level
–
the
longest
established
and
experienced,
where
most
development
cooperation
interventions
still
take
place
-‐
is
the
project
level.
At
the
project
level
result
frameworks
(such
as
a
Logical
Framework
Approach)
support
an
individual
project.
The
third
level
-‐
where
a
significant
amount
of
development
has
taken
place
during
the
past
decade
-‐
is
the
partner
country
level.
At
this
level,
development
cooperation
has
been
provided
in
the
form
of
capacity
development
to
improve
partners’
own
performance-‐based
systems,
or
within
a
larger
donor-‐partner
country
program
–
such
as
budget
or
programme
support
to
a
country’s
Poverty
Reduction
Strategy
or
Sector
Strategy
,
aligned
to
the
country’s
financial
and
performance
management
systems..
We
have
used
the
following
literature
as
empirical
background
for
the
levels
listed
above:
1. Evaluations
and
reviews
conducted
of
donor
RBM-‐
systems:
a)
A
review
of
experience
of
RBM
in
Development
Cooperation
Agencies.
The
review
covered
USAID,
Australian-‐AusAID,
Danish
-‐DANIDA,
United
Kingdoms
-‐DFID,
Canadian
-‐
CIDA,
World
Bank,
and
the
United
Nations
Development
Programme-‐UNDP.
Submitted
by
OECD/DAC
Working
Party
on
Evaluation
and
conducted
by
Annette
Binnedjikt
(2000)
b)
Evaluation
of
RBM
at
the
United
Nations
(2008)
c)
Evaluation
of
RBM
at
the
UNDP
(2007)
d)
A
Review
of
RBM
in
multilateral
development
Institutions,
“Easier
Said
than
done”.
The
review
covered
UNDP,
UNICEF,
UNIFEM,
Interamerican
Development
Band
–IDB,
and
the
World
Bank-‐
WB
and
was
conducted
by
Michael
Flint
(2003).
e)
Evaluation
of
Result
Based
Approach
by
Ministry
of
Foreign
Affairs
in
Finland
(2011)
f)
Evaluation
of
results
management
in
Swedish
Development
Cooperation,
by
Statskontoret
(2011)
h)
Andrew
Natsios
paper
on
“The
clash
of
counterbureaucracy”
–about
implementation
of
RBM
at
the
USAID
(2010)
g)
Management
for
Development
Results
Sourcebook
by
the
OECD/DAC
with
some
examples
on
the
implementation
of
RBM
in
some
donor
agencies,
such
as
the
Canadian
International
Development
Agency
(Cida)
(2005)
h)
Report
on
Donor
Result
Reporting
by
Gilroy
Management
with
experiences
from
USA,
Germany
and
UK
(2011)
i)
A
summary
report
from
OECD/DAC
on
DAC
Peer
Reviews,
with
experiences
summarised
from
all
OECD/DAC
members
(2011)
2. Experiences
from
project
level.
Reports
and
evaluations
that
led
to
the
Paris
Declaration,
and
examples
often
related
by
the
OECD/DAC
but
also
by
individual
donors
on
practices
that
try
to
implement
a
result
focus
at
the
project
level,
including:
a)
selected
DAC
Peer

19
Reviews
1998,
b)
DAC
sourcebooks
on
Managing
for
development
results
2005,
2006,
2007,
2008,
c)
INTRAC
2005,
d)
Forrester
and
Macloughlin
(1998),
e)
Sida
internal
evaluations
2004
and
2006.
3. Evaluations
and
reports
from
implementation
of
a
results
focus
in
partner
countries
and
larger
programmes,
including:
a)
OECD/DAC
Progress
in
implementing
the
Paris
Declaration
2011,
b)
The
evaluation
of
Budget
support
conducted
by
IDD
and
Associates
(2006)
that
evaluates
experiences
of
General
Budget
Support
in
Burkina
Faso,
Malawi,
Mozambique,
Nicaragua,
Rwanda,
Uganda,
Vietnam
c)
WB-‐report
on
results
oriented
national
development
strategies
(2007)
d)
Independent
Evaluation
Group
(IEG)
of
the
World
Bank,
Annual
Review
of
Aid
Effectiveness,
2008
In
some
parts,
we
have
also
used
articles
and
other
forms
of
research
to
illustrate
the
findings.
In
general,
the
management
chains
can
be
numerous
and
quite
complicated
in
development
cooperation.
Our
previous
experience
tells
us
that
it
is
not
unusual
that
management
chains
constitute
between
5-‐7
relations.
The
following
picture
attempts
to
illustrate
the
different
management
chains
for
a
Sida
funded
Civil
Society
support
to
a
partner
country
trade
union:
Obviously,
there
exist
a
number
of
difficulties
in
attributing
results
and
possibilities
for
the
principal
financier
to
steer/control
results
achieved
to
the
lowest
level.
Also
the
challenges
are
perhaps

greatest
for
actors
in
the
middle
of
the
chain
because
they
stand
in
between
instructions
on
results
to
be
achieved,
and
pushing
for
results
to
be
achieved
without
sufficient
control.
20
16
5.1. Experiences from donor institutional level
Donor
organisations
have
understood
and
adapted
RBM
in
varying
ways.
Based
on
the
literature
reviewed,
we
have
found
that
the
challenges
and
lessons
faced
in
the
implementation
of
RBM
are
quite
similar
for
donor
organisations,
be
they
multilateral
or
bilateral
agencies.
As
a
result,
we
have
only
used
information
provided
in
evaluations
and
reviews
on
how
the
application
of
the
model
has
succeeded
(according
to
the
respective
organisation’s
understanding
of
the
model).
In
this
chapter,
we
will
only
use
evaluations
and
reviews
regarding
experience-‐based
challenges
and
successes.
While
most
of
the
experiences
refer
to
challenges,
a
few
successes
are
also
noted.
In
general,
the
literature
implies
very
weak
positive
effects
from
the
application
and
implementation
of
RBM.
All
evaluations
and
reviews
do,
however,
provide
recommendations
for
improvement,
with
the
inclusion
of
experience
from
lessons
and
“best
practices”.
Since
the
recommendations,
lessons
and
best
practices
are
more
based
on
an
ideology
of
how
RBM
ought
to
work,
and
not
actually
based
on
experience,
we
have
separated
and
categorised
them
in
chapter
8.
General
challenges
in
implementation
encountered
in
the
evaluations
and
reviews
are:
Shortcomings in the original design of the model
All
of
the
reports
raise
the
issue
of
the
challenges
with
the
application
of
the
Result-‐Based
Model
itself.
In
most
organisations
the
method
is
translated
into
results
frameworks
used
at
different
organisational
levels
(country
level,
project
level,
programme
level,
and
individual
performance
level).
We
have
not
analysed
to
what
extent
organisations
attempt
to
attribute
results
from
one
level
to
another,
but
it
seems
as
if
some
organisations
have
experimented
with
attributing
for
example
project-‐level
results
to
the
results
in
the
organisation’s
country
strategy
(e.g.
Sida).
Some
organisations
try
to
attribute
internal
and
external
results
–
for
example
that
an
individual
manager’s
performance
is
judged
by
the
performance
of
the
project
that
the
manager
is
responsible
for
(e.g.
USAID).
Most
organisations
do
not
seem
to
correlate
funding
decisions
to
projects/programmes
with
performance.
However,
organisations
such
as
DfiD
have
introduced
performance
contracts
with
their
field
offices,
where
funding
to
the
country
portfolio
is
connected
to
results
in
the
country
programme.
However,
it
seems
that
many
organisations
are
still
at
the
early
stages
of
developing
management
models
where
greater
attribution
is
made
between
the
different
levels.
Probably
the
most
cited
challenge
related
to
the
method/model
itself
is
the
difficulty
in
selecting
appropriate,
objective
indicators
that
measure
relevant
results
rather
than
activities.
However,
it
16
An
interesting
blog
post
(in
Swedish)
on
this
topic
is
written
by
Kajsa
Johansson,
representative
of
a
Swedish
CSO
in
Mozambique,
who
writes
about
“My
life
as
a
hamburger”,
describing
how
the
organisation
is
squeezed
between
result
requirements
from
the
top
levels,
and
with
the
task
to
see
to
it
that
results
are
achieved
and
reported
by
local
level
organisations
with
very
limited
experience
with
RBM.
http://kajsajohansson.wordpress.com/2010/01/13/mitt-‐liv-‐som-‐hamburgare/
Also,
Curtis
(2004),
discusses
this
topic
from
a
consultant’s
point
of
view.
He
states
that
due
to
power
relations
in
development
cooperation,
the
donor
agent
controls
the
resources,
and
the
message
becomes
rules
of
access
and
performance.
The
broker
(in
this
case
the
consultant)
is
left
to
make
the
most
of
the
situation,
looking
for
room
for
maneuver
-‐
sometimes
in
vain.

seems
as
if
the
selection
of
appropriate
indicators
could
be
easier
in
some
sectors,
and
specifically
in
sectors
such
as
education,
where
there
exist
jointly
agreed
upon
global
monitoring
indicators,
for
example
the
Education
For
All
Framework.
There
seems
to
be
general
difficulty
in
managing
for
outcomes
rather
than
outputs,
and
establishing
a
link
between
inputs
to
impacts
across
the
results
chain.
Many
reviews
are
critical
that
planned-‐for
results
lack
credible
methods
for
verification,
and
that
performance
measures
lack
baselines
and
targets
(UN,
UNDP).
Also
commonly
cited
are
difficulties
in
aggregating
data
and
attributing
results
to
aid-‐funded
projects
and
programmes.
The
MfDR-‐sourcebook
concludes
that
results
frameworks
tend
to
become
overly
complex
with
an
excessive
numbers
of
indicators,
and
despite
adherence
to
the
notion
of
using
a
results
framework
as
a
“living
management
tool”,
that
it
is
difficult
to
keep
the
framework
sufficiently
lightweight.
Some
reviews
bring
up
time-‐aspect
difficulties,
e.g.
the
unrealistic
goal
of
actually
tracking
outcomes
on
an
annual
basis
(Sida,
USAID,
UN).
Natsios,
a
former
director
at
USAID,
argues
that
the
search
for
quick
outcomes
has
diverted
attention
to
more
easily
measured
outcomes
at
the
expense
of
those
less
easily
observed
or
counted;
and
that
this
practice
has
led
to
more
funding
being
provided
for
development
sectors
in
hard
sciences,
such
as
public
health
and
medicine,
and
less
to
softer
sciences
such
as
democracy
and
governance
programmes.
However,
despite
all
challenges
the
OECD/DAC
summary
report
from
DAC
Peer
Reviews
has
determined
that
most
DAC-‐members
find
that
the
stronger
focus
on
results
-‐
inherent
in
the
aid
effectiveness
principles
-‐
helps
to
broaden
support
for
aid,
and
that
it
also
helps
drive
a
cultural
change
in
how
the
programmes
are
managed.
In
the
wake
of
the
Millennium
and
Paris
Declarations
some
donors
also
seem
to
report
progress
towards
result
orientation17.
21
Difficulties with non-usage for the original purposes
Many
of
the
evaluations
and
reviews
report
on
difficulties
where
results
information
is
not
being
used
for
improved
decision-‐making,
budgeting
and
prioritisation
purposes,
and/or
strategic
and
policy
debate.
The
UN-‐review
strongly
concludes
that
“RBM
in
the
United
Nations
has
been
an
administrative
chore
of
little
value
to
accountability
and
decision-‐making”
and
that
“results-‐based
management
processes
have
been
found
to
have
made
virtually
no
contribution
to
strategic
decisions
in
any
of
the
reviewed
organizations”.
The
findings
of
the
Gilroy
report
-‐
that
reviewed
12
donor
reports,
and
the
prevalence
of
differing
RBM
elements
(baselines,
targets,
result-‐chains
etc)
-‐
are
a
bit
more
positive.
It
found
that
some
of
the
reviewed
results
reports
were
indeed
used
for
learning
and
management
purposes.
Yet,
it
also
found
that
only
one
report
out
of
twelve
reviewed
had
all
of
the
RBM
elements
(baselines,
targets,
result-‐chains,
etc.)
present,
while
an
additional
four
of
the
reports
reviewed
had
most
of
the
RBM
elements.
Several
of
the
documents
discuss
why
results
information
is
not
used.
Reasons
relate
to
the
original
design,
difficulties
with
tracking
causal
linkages,
and
also
difficulties
in
measurement
and
data-‐
collection.
Several
reviews
also
point
to
tensions
between
using
the
information
for
management
17
This
seem
to
be
true
at
least
for
Cida,
Dfid,
WB
and
ADB
–
according
to
Peer
Reviews
and
their
own
assessment.

improvement,
or
for
accountability
reporting
(DAC,
Multi-‐review).
The
UN-‐evaluation
points
to
the
fact
that
RBM
has
been
introduced
as
an
add-‐on
to
pre-‐existing
rules
and
procedural
requirements,
and
that
a
lack
of
relaxation
in
other
procedures
has
caused
more
administrative
burdens.
Non-‐usage
is
also
attributed
to
a
lack
of
leadership,
staff
motivation,
and
incentives
to
use
the
results.
The
UN-‐
review
concludes
that
there
are
no
consequences
for
non-‐compliance
or
non-‐attainment
of
results.
Other
explanations
for
non-‐use
of
results
information
can
be
found
in
the
following
sections
and
chapters.
22
Difficulties in application in development cooperation as such
In
addition
to
the
experiences
mentioned
in
a-‐c,
which
seem
to
be
quite
similar
to
experiences
faced
in
other
public-‐sector
agencies18,
the
introduction
of
RBM
in
development
cooperation
seems
to
face
some
specific
challenges.
According
to
Binnedjikt,
the
establishment
of
performance
measurement
systems
can
be
more
complex
and
costly
in
development
cooperation
than
in
other
public
agencies.
Aid
agencies
work
in
many
different
countries
and
contexts,
where
the
causality
chains
between
funded
activities
and
possible
outcomes
and
impacts
are
exceptionally
long
and
complex.
In
addition,
donors
need
to
rely
on
outcome
and
impact
data
collected
by
the
partners.
These
partners,
whether
within
government,
academia,
or
the
private
sector,
have
limited
technical
capacity
and
resources.
Furthermore,
they
normally
lack
registry
data
of
population,
taxes,
transfers,
etc.
that
can
be
found
in
developed
countries,
and
is
often
used
for
data
analysis
and
sampling.
Reliable
qualitative
data
is
non-‐existent
in
many
countries.
Further
confounding
the
problem
is
that
development
cooperation
often
involves
areas
where
results
are
difficult
to
measure
–
such
as
capacity
building,
institutional
and
policy
reforms,
and
good
governance.
Most
evaluations
and
reviews
point
to
the
difficulties
of
building
RBM
systems
at
the
donor
institutional
level
due
to
weak
development
country
capacity,
the
influence
on
outcomes
by
multiple
unchecked
actors,
and
external
risk
factors,
etc.
The
weak
development
country
capacity
leads
to
donors
not
using
results-‐oriented
and
monitoring
frameworks
in
partner
countries,
which
in
turn
prevents
such
systems
from
being
improved.
The
evaluation
of
RBM
in
Swedish
Development
cooperation
states
that
there
are
too
many
conflicting
goals
in
results
frameworks
at
different
levels
in
the
organisation
which
makes
it
difficult
for
managers
to
know
what
to
prioritise,
and
subsequently
for
the
priorities
of
the
government
to
be
fulfilled.
Since
development
cooperation
activities
operate
in
so
many
policy
areas,
and
have
so
many
actors
and
interests,
it
seems
as
though
the
amount
of
potential
goals
to
be
fulfilled
are
larger
than
in
other
policy
areas.
Conflicting purposes
Several
of
the
reports
refer
to
conflicting
purposes
when
introducing
RBM
in
development
cooperation.
The
following
conflicting
purposes
are
commonly
cited
in
the
literature:
a)
The
dual
role
of
introducing
RBM
internally,
while
at
the
same
time
supporting
partner
country
governments
in
the
implementation
of
RBM
reform
(Multi-‐Review,
Binnedjikt);
b)
the
conflict
between
result
reporting/accountability
on
the
one
side,
and
the
use
of
information
for
learning/improved
18
See
for
example
Sundström
(x)
and
Lindgren
(y)
and
Ordonez
(2009)

management
on
the
other
side;
c)
the
need
to
demonstrate
results
that
can
lead
to
risk-‐averse
behaviour
versus
the
need
to
engage
in
important
initiatives
that
have
less
quantifiable
outcomes,
or
entail
higher
risks
(PD-‐Evaluation,
DAC
Peer
Review
report,
Natsios
etc.).
Another
conflicting
purpose
(not
directly
cited
in
the
evaluation
literature,
but
based
on
experience)
is
the
conflict
between
the
disbursement
goal/pressure
and
results.
As
for
the
first
conflict,
between
partner
countries
and
donor
results
interests,
our
review
of
the
literature
confirms
that
many
donors
seem
to
face
a
tension
between
reporting
on
domestic
goals,
and
aligning
with
the
interest
of
partners.
The
DAC
Peer
review
summary
report
addresses
that
one
of
the
largest
challenges
for
donors
is
selecting
and
using
indicators
that
are
consistent
with
partners’
national
development
strategies
and
monitoring
frameworks.
Many
agencies
still
feel
the
need
to
attribute
results
to
the
respective
country/donor
intervention,
whereas
clear
attribution
is
usually
not
possible.
The
domestic
demand
for
accountability
(from
domestic
constituencies,
parliaments,
senior
management,
executive
boards
etc.)
is
also
usually
difficult
to
combine
with
an
agenda
of
supporting
partner-‐country
priorities
and
interventions.
The
MfDR-‐review
states
that
there
are
currently
only
a
few
examples
of
donors
that
have
actually
been
prepared
to
“lower
the
flag”,
and
suspend
their
individual
procedures
in
favour
of
joint
strategies,
decreasing
the
number
of
individual
voices
at
the
table,
and
becoming
silent
partners
represented
by
other
donors.
With
regard
to
the
second
conflict
–
the
use
of
RBM
for
external
accountability
or
learning
purposes
–
our
review
of
the
literature
confirms
that
results
information
seem
to
mainly
have
been
used
for
the
purpose
of
accountability
needs/reporting
purposes
in
donor
countries,
and
less
so
for
learning
purposes
within
the
organisations.
As
for
the
third
conflict,
for
example
Andrew
Natsios
claims
that
the
results
agenda
has
created
incentives
that
threaten
good
development
practice.
Natsios
uses
the
term
for
this
phenomenon
as
it
threatens
a
central
principle
of
development
theory
-‐
that
those
development
programmes
that
are
most
precisely
and
easily
measured
are
the
least
transformational,
and
those
programmes
that
are
most
transformational
are
the
least
measurable.
He
also
claims
that
the
RBM
agenda
within
USAID
has
led
to
clashes
between
compliance
officers
and
technical
programme
specialists
(in
health,
agriculture,
education,
etc.)
over
attempts
to
measure
and
account
for
everything;
and
that
this
has
led
to
more
risk-‐averse
behaviour,
and
the
compromise
of
good
development
practices.
Similar
warning
signals
of
these
conflicts
are
raised
in
the
evaluation
reports
–
that
these
conflicting
purposes
can
disempower
operational
managers
and
partners,
stifle
innovation
and
flexibility,
create
adverse
incentives,
etc.
Some
have
reported
that
these
conflicting
purposes
are
strong
enough
to
be
counterproductive.
The
UNDP-‐evaluation
for
example
concludes
that
“The
introduction
of
corporate
systems
and
tools,
which
have
had
some
efficiency
benefits,
have
not,
however,
strengthened
the
culture
of
results
in
the
organization
or
improved
programmatic
focus
at
the
country
level.”
Furthermore,
the
UN-‐evaluation
determines
that
the
overly
formalistic
approach
to
codifying
outcomes
has
stifled
the
innovation
and
flexibility
required
to
achieve
those
outcomes.
The
evaluation
thus
concludes
that
“the
determination
of
development
success
does
not
lend
itself
to
impartial,
transparent
and
precise
measurement”.
23

24
Lack of human resources, capacity and incentives
Many
of
the
challenges
expressed
in
the
evaluations
relate
to
a
lack
of
human
capacity,
knowledge,
or
incentives
to
fulfil
the
complex
requirements
demanded
by
an
RBM-‐approach.
Many
reports
seem
to
raise
the
issue
that
officials
within
the
agencies
no
longer
have
time
for
implementing
programmes,
and
that
a
lot
of
time
and
efforts
is
wasted
on
data
collection,
reporting,
and
control.
Some
reports
have
addressed
a
shortage
of
time
to
do
analysis
of
the
indicators,
and
that
there
are
not
sufficient
support
mechanisms
for
the
analysis.
Binnedjikt
notes
that
officers
claim
that
data
collection
is
not
considered
directly
relevant
or
useful,
and
is
only
being
used
to
“report
upward”.
Furthermore,
there
is
too
much
concern
over
indicator/data
validity,
which
may
limit
its
practical
utility
as
a
motivational
and
management
tool.
The
Finnish
evaluation
concludes
that
it
is
not
the
quality
of
the
guidelines
that
is
at
fault.
The
evaluation
claims
that
there
is
too
little
competence
and
understanding
of
the
method
within
the
organisation.
This
conclusion
is
similar
to
that
of
the
Swedish
evaluation
–
that
different
actors
within
the
management
chain
have
difficulties
understanding
the
system.
In
contrast,
the
OECD/DAC
has
come
to
the
conclusion
that
donors
with
adequate
human
and
financial
resources,
and
who
have
placed
a
strong
emphasis
on
maintaining
cohesion
and
ensuring
quality
across
the
programme,
obtain
better
aid
results
(Peer
Review
Summary
Report).
5.2. Experiences from project level
Struggle with results oriented project development tools
As
regards
project
mode
development
cooperation
the
literature
suggests
a
general
struggle
with
results
oriented
project
development
tools
(LFA,
GOPP
etc.).
Recurring
themes
include
1. The
tools
are
too
“project
specific”
in
scope
to
influence
overall
planning,
and
2. The
tools
are
too
“limited
from
a
management
perspective”
to
challenge
prevailing
management
practices,
which
are
generally,
but
not
always,
process
and
control
oriented.
It
has
been
suggested
that
organisational
cultures
are
unlikely
to
be
affected
by
isolated
or
limited
efforts
to
change
them.
In
sum,
no
one
seems
to
argue
that
project-‐level
results
management
has
had
much
effect
on
donor
or
partner
organisational
behaviour.
On
the
contrary,
repeated
donor
internal
evaluations
and
DAC
peer-‐reviews
suggest
that
rather
than
stimulating
a
results
focus
in
developing
aid
management,
results
oriented
tools
and
matrices
at
the
project
level
have
been
regarded
as
requirements,
and
add-‐ons,
to
activity
or
process
oriented
management
practices.
(Sida
2004,
DAC
Peer
Reviews
1998-‐2009)
Perhaps
the
most
persistent
concern
is
that
project
level
management
tools
(such
as
LFA)
have
not,
overall,
been
used
as
prescribed.
In
LFA,
the
idea
is
that
managers
should
be
vigilant
to
the
development
of
risk
factors
and
results
indicators,
and
come
up
with
a
management
response
if
required.
Instead,
it
has
been
suggested
that
the
LFA
matrix,
because
it
has
been
neglected
during
implementation
and
regarded
much
as
a
prescription/requirement,
has
rather
perpetuated
static
implementation
procedures
(INTRAC
2005).
One
way
of
looking
at
this
is
by
observing
the
traditional
relationship
between
financier
and
implementer
of
projects;
using
the
matrix/tool
as
an
instrument
of
assessment
of
the

25
project/implementing
agents
rather
than
as
a
living
instrument
of
learning
and
change
in
view
of
performance
or
altered
conditions
(INTRAC
2005,
Sida
2004,
2006).
Too many fragmented efforts – limited effects on overall level
Project
mode
cooperation
also
has
a
history
of
supporting
its
own
“project
internal”
information
needs.
Project
reporting
is
striking
because
of
the
overwhelming
abundance
of
information
directly
linked
to
implementation
(the
input-‐activity-‐output
levels
of
the
results
chain).
In
larger
projects,
the
need
for
contextual
information
has
generally
been
viewed
as
greater;
and
in
particular,
the
multilateral
banks
have
a
record
of
supporting
monitoring
and
evaluation
capacities
with
local
counterparts.
At
times,
the
demand
for
information
has
covered
development
patterns
and
behavioural
changes
among
populations
and
organisations
outside
the
realm
of
the
project,
thus
leading
to
efforts
to
support
national
statistical
systems.
However,
in
part
because
of
un-‐coordinated
donor
activities,
systems
development
has
also
been
un-‐
coordinated.
Weak
national
systems
have
been
vulnerable
to
the
introduction
of
inconsistent
methodologies,
terminologies,
and
approaches
to
systems
development
as
initiated
by
a
variety
of
donors.
There
is
an
ongoing
initiative
-‐
PARIS21
-‐
that
has
repeatedly
pointed
towards
these
conclusions,
and
the
need
for
more
concerted
efforts
(PARIS21,
2009).
Successful when adapted to local conditions
Most
project
results
management
tools
are
concerned
with
a
participatory
process
of
identification,
and
joint
agreement
on
objectives
and
indicators
at
the
beginning
of
operations.
In
this
capacity,
they
are
often
reported
as
having
had
a
positive
impact
on
project
logic.
When
application
is
successful,
ensuing
strengths
are
shared
objectives
between
stakeholders
and
a
real
adaptation
to
local
conditions.
At
the
project
level,
recorded
positive
experiences
are
mostly,
but
not
always,
related
to
larger
projects
and
countries
outside
the
group
of
the
Least
Developed
Countries
(LDCs).
For
example,
increased
results
management
have
been
reported
to
improve
ownership,
and
in
aligning
planning
and
reporting
with
national
systems.
(OECD/DAC
Sourcebooks
2005,
2007
and
2008)
A
typical
example
is
reported
by
Simon
Delay
in
his
article
“Why
does
Aid
Management
in
Samoa
succeed?”,
which
concludes
that
aid
management
in
Samoa
is
unusually
successful
for
the
region.
The
success
factors
owe
to
the
involvement
of
high-‐level
leadership
(donor
coordination
is
chaired
by
the
Prime
Minister)
in
aid
coordination
and
the
capacity
and
stability
of
key
staff.
He
declares
that
there
is
a
well
functioning
aid
coordination
system
in
Samoa,
and
that
the
government
has
taken
a
clear
lead
over
the
process
–
“the
government
is
unusually
willing
to
articulate
its
desires
and
to
reject
approaches
and
individual
activities
which
did
not
fit
with
its
priorities.”
Moreover
the
success
depends
on
the
integration
of
donor
project
planning
and
reporting
with
national
planning
processes.
Technical
assistance
has
been
used,
but
Samoa
has
maintained
ownership
over
the
process.
In
2008
the
IEG
concluded
that
in
World
Bank
supported
projects,
there
is
a
positive
relationship
between
project
achievement
and
the
strength
of
the
monitoring
and
evaluation
system.
Importantly,
the
strength
of
the
monitoring
and
evaluation
system
was
characterised
by
its
level
of

26
outcome
orientation.
IEG
observed
a
positive
correlation
between
ratings
of
project
outcome
and
monitoring
and
evaluation
systems.
More
outcome
related
monitoring
and
evaluation
systems
resulted
in
better
project
outcome
level
performance.
An
inverse
relationship
was
also
confirmed
(IEG
2008,
pp.
21-‐23).
It
is
important
that
partners
agree
on
successful
results
orientation
leading
to
better
dialogue,
better
planning,
ownership
and
alignment.
Perhaps
more
importantly,
outcome
performance
may
be
positively
related
to
the
strengths
of
the
monitoring
system
(defined
by
its
level
of
results
orientation).
But
these
findings
do
not
address
the
issues
of
project-‐level
fragmentation
as
an
inherent
challenge
to
a
results
management
perspective
in
development
cooperation.
5.3. Experiences from partner country and programme level
Few experiences from partner country implementation
Results-‐oriented
planning
and
learning
in
developing
countries
has
been
described
as
weak
throughout
the
history
of
development
cooperation.
A
key
aspect
has
been
the
relative
absence
of
analytical
and
overall
performance
information
in
any
particular
field.
Without
such
information,
the
foundation
for
strategic
planning
and
reporting
is
weakened.
A
variety
of
explanations
have
been
offered
-‐
including
cultural,
educational,
traditional
and
political.
For
a
long
time,
partner
country
experiences
resembled
those
at
the
level
of
project
cooperation.
Partner
country
systems
for
performance
monitoring
and
analysis
remain
weak
in
project-‐mode
due
to
development
objectives
and
reforms
being
defined
(foremost)
externally,
and
reporting
being
centred
on
project
performance
rather
than
on
national
or
sector
developments.
Within
this
context,
low
levels
of
national
demand
have
not
helped
the
availability
and
use
of
quality
information.
Early programme experience: still focus on activity and input
During
the
1980-‐90s,
with
many
development
cooperation
interventions
moving
slowly
towards
larger
and
more
comprehensive
programmes
and
policy
processes
(a
larger
part
still
continued
to
be
project-‐based19),
the
full
scale
of
this
lack
of
information
was
brought
to
light.
One
experience
with
the
programmes
of
the
1990s
and
early
2000s
is
that
they
initially
emphasised
joint
donor
management
(and
harmonising
procedures)
before
turning
their
attention
to
issues
of
monitoring
and
evaluation.
(See
for
instance
WB
2007,
pp.
24-‐25)
At
the
turn
of
the
century,
development
reports
overwhelmingly
complained
of
a
lack
of
information,
and
the
need
to
address
the
issue.
Not
only
had
project
mode
cooperation
emphasised
budget,
activity
and
output
information,
but
the
early
programmes
had
too.
One
spoke
of
a
“gap”,
and
that
gap
consisted
of
a
lack
of
knowledge
about
“the
situation
on
the
ground”,
or
outcome
level
information
in
RBM
terminology
(EU
2002).
19
During
this
time
different
sources
gave
different
accounts
of
the
balance
between
project
and
programme
support,
but
on
the
whole
suggested
project
aid
as
the
dominating
source
of
finance.
More
recently,
the
follow-‐up
of
the
Paris
Declaration
has
indicated
that
in
2011
45%
of
all
ODA
was
delivered
through
programme
based
approaches
(PBA).
Between
2005
and
2010
the
proportion
of
PBA
in
the
baseline
group
of
countries
(32)
had
risen
from
42
to
48%,
way
clear
of
the
target
of
66%.
See
OECD/DAC
(2011),
Aid
Effectiveness
2005-‐2010,
p.
63.

27
New programme efforts during 2000: Remaining substantial challenges
So,
entering
the
age
of
the
Millennium
and
Paris
Declarations,
partner
countries
had:
a. weak
and
non-‐coherent
information
collection
systems
b. poor
national
demand
for
information
and
analysis
c. weak
national
planning
frameworks
(in
part
because
of
the
lack
of
information)
The
suggested
remedy,
as
stated
in
the
Paris
Declaration,
was
to
align
with,
use,
and
strengthen
national
policy
frameworks
based
on
reliable
national
performance
assessment
frameworks.
The
Declaration
suggested
that
Programme-‐Based
Approaches
would
be
the
modality
for
implementing
these
principles.
Several
of
such
approaches
have
been
launched
during
the
last
decade,
and
the
programmes
have
been
built
on
joint
results
frameworks
aligned
to
partner
country
strategies.
However,
the
monitoring
and
evaluation
of
the
Paris
Declaration
has
shown
how
difficult
it
is
to
successfully
strengthen
national
performance
assessment
frameworks.
The
World
Bank
concluded
in
2007
that
progress
had
been
made
in
all
their
partner
countries,
but
that
substantial
challenges
remained.
Progress
is
best
developed
in
countries
who
have
come
farthest
in
developing
their
own
policy
frameworks
(as
evident
through
their
use
of
second
generation
Poverty
Reduction
Strategies,
according
to
the
World
Bank).
And
progress,
in
this
case,
is
judged
by
increasing
policy
coherence
(unification
of
the
strategic
framework),
more
rational
budget
priorities,
and
an
improved
policy
dialogue
both
with
and
between
donors
and
line
ministries/national
actors
(WB
2007,
pp.
14-‐19).
The
Joint
Evaluation
on
General
Budget
Support,
conducted
by
IDD
(International
Development
Department)
and
Associates
finds
evidence
that
the
modality
has
been
effective
in
strengthening
public
finance
management,
including
planning
and
budgeting,
in
partner
countries.
It
also
concludes
that
results
delivery
–
particularly
service
delivery
in
some
sectors,
such
as
health
–
has
been
more
effective
if
supported
through
general
budget
support.
An
important
aspect
of
the
challenges
reported
by
the
World
Bank
is
frail
linkages
between
performance
assessment
frameworks
and
sector
planning
and
budgeting
processes.
Partner
country
planning
is
often
described
as
vertical,
with
limited
horizontal
linkages
between
different
planning
units,
and
poor
use
of
common
information
sources.
Rectifying
the
situation
has
been
a
central
theme
of
the
Paris
Declaration
and
its
follow
up.
The
2011
OECD
Report
on
Progress
in
Implementing
the
Paris
Declaration
2005-‐2010
draws
a
number
of
conclusions
regarding
the
efforts
to
strengthen
results
management.
The
overall
conclusion
is
that
of
the
five
principles
of
the
Declaration,
this
is
probably
where
progress
is
least
visible.
On
the
level
of
partner
countries,
the
report
concludes
that
the
development
of
national
performance
assessment
frameworks
has
been
slow,
though
not
insignificant.
Importantly,
it
notes
that
national
systems
are
commonly
strengthened
where
donor/recipient
partnerships
favour
the
use
of
performance
information
in
planning
and
reporting
-‐
which
is
still
relatively
rare.
More
specifically,
“the
use
of
performance
indicators
that
are
consistent
with
partners’
national
development
strategies
and
reporting
and
monitoring
frameworks
remains
infrequent.
Some
donors
tend
to
focus
on
funding
and
activities
(inputs/processes)
rather
than
results
on
the
ground…”
(pp.
89-‐90).

Observations
of
this
nature
are
confirmed
by
continuous
analyses
made
by
the
IEG.
At
the
country
level,
IEG
concludes
that
both
donors
and
national
authorities’
incentives
favour
activity
preparation
and
delivery
(output)
over
monitoring
and
evaluation.
Also,
national
systems
development
still
tends
to
emphasise
process,
input,
and
output
while
disregarding
outcome.20
These
latest
findings
seem
to
confirm
that
country-‐level
results
management
comes
about
primarily
due
to
a
strong
donor
influence.
They
also
seem
to
confirm
that
earlier
findings
about
both
donor
and
partner
management
practices
being
focused
on
input
and
activities
(rather
than
outcomes)
may
still
be
valid.
As
the
case
may
be,
performance
information
still
appears
to
be
underutilised
in
national
planning
and
policy
making.
28
Some progress on statistical systems development and transparency
The
OECD
monitoring
report
on
the
Paris
Declaration
(2011)
goes
on
to
conclude
that
the
development
of
statistical
systems
to
supply
development
related
information
has
improved,
although
slowly,
and
with
considerable
regional
differences.
In
parallel,
World
Bank
evaluations
indicate
that
progress
has
been
most
prominent
in
sectors
(in
particular
the
health
sector)
with
increasing
donor
financing.21
Although
overall
progress
has
been
made
on
Paris
Declaration
indicator
11
-‐
the
development
of
transparent
and
monitorable
performance
assessment
frameworks
-‐
caution
is
invited
with
regard
to
progress
on
indicator
12
on
mutual
accountability.
The
indicator
on
mutual
accountability
combines
several
aspects22
that
touch
upon
a
national
and
joint
ability
to
demonstrate
the
use
of
performance
information
in
planning,
and
to
communicate
joint
results
to
constituencies
and
the
public.
Here,
the
OECD/DAC
concludes
that
progress,
to
date,
has
been
questionable.
It
is
reported
that
donors
are
obscuring
results
clarity
by
introducing
several
layers
of
attribution
between
the
partner
countries’
and
their
own
goal
structures,
and
by
setting
up
their
own
non-‐outcome
oriented
reporting
systems
instead
of
strengthening
national
systems.
23
Against
this
background,
it
seems
that
contextual
factors
constrain
the
strengthening
of
partner
country
systems
and
capacities
with
results
management.
In
partner
countries,
a
traditional
demand
for
quality
information
to
underpin
planning,
policy,
and
dialogue
is
weak.
The
tendency
of
donors
towards
uncoordinated
behaviour
centred
on
their
own
reporting
on
budgets,
activities
and
easy-‐to-‐
measure
outputs
has
inhibited
change.
To
summarise
our
literature
review
on
experiences
from
the
implementation
of
RBM
at
the
donor,
organisational,
project,
and
program
levels,
we
find
that
the
implementation
of
RBM
at
the
donor
level
has
encountered
challenges
due
to
(1)
conflicting
purposes
of
introducing
the
method,
(2)
specific
difficulties
in
applying
the
method
in
development
cooperation
as
such,
and
(3)
difficulties
with
regard
to
resources
and
a
capacity
for
implementation.
20
IEG
2008,
pp.
26-‐28.
21
PARIS21
(2009a),
PARIS21
at
Ten
–
Improvements
in
Statistical
Capacity
since
1999,
PARIS21,
Paris.
22
For
a
full
review
compare
OECD/DAC
(2011),
Aid
Effectiveness
2005-‐2010;
Progress
in
implementing
the
Paris
Declaration.
23
See
OECD/DAC
(2011),
Aid
Effectiveness
2005-‐2010;
Progress
in
implementing
the
Paris
Declaration,
pp.
89-‐
91.

29
At
the
project
level,
we
find
little
evidence
of
success
in
the
application
of
different
project
development
tools,
such
as
the
LFA.
However,
there
are
individual
success
stories
from
projects.
With
the
new
set
of
programmes,
such
as
budget
and
programme
support,
the
literature
points
to
evidence
of
improved
public
financial
management,
planning
and
budgeting
procedures,
and
to
improved
service
delivery
in
some
sectors.
It
also
points
to
improved
performance
assessment
systems
in
countries
with
a
strong
policy
framework
for
poverty
reduction.
The
OECD/DAC
also
reports
that
a
larger
results
focus
has
improved
public
support
for
aid,
and
has
changed
the
culture
of
how
aid
programmes
are
managed.
This
is
in
line
with
the
experiences
preceding
the
Paris
Declaration
and
the
Millennium
Declaration
Goals
–
that
a
focus
on
goals
and
results
is
fundamental
to
improving
management
practices
to
actually
fulfil
development
objectives.
Our
review
of
the
literature,
however,
points
to
a
recurring
and
persistent
problem
with
how
organisations
apply
RBM
in
practice.
For
the
following
chapter,
we
have
reviewed
theories
that
could
help
us
to
explain
why
the
application
of
RBM
has
encountered
so
many
difficulties.

30
6. Some theoretical explanations
So
far,
we
have
concluded
that
there
exist
several
challenges
to
the
implementation
of
results-‐based
management,
and
that
there
have
also
been
some
successes.
This
chapter
will
shortly
summarise
research,
from
different
disciplines,
that
discusses
and
provides
alternative
ways
to
look
at
development
cooperation
relations,
and
how
such
relations
shape
decision-‐making
processes.
6.1. Complexity theory
Over
the
past
few
years,
many
development
cooperation
papers
and
research
on
development
issues
have
been
grounded
in
complexity
science
or
complexity
theory
(see
for
e.g.
Ramalingam
and
Jones).
In
general,
complexity
theory
is
critical
of
the
current
paradigm
of
results
management,
and
in
particular
it
questions
its
emphasis
on
linear
causality
and
accountability
through
top-‐down
command
and
control.
Complexity
theory
seem
to
posit
that
self-‐organising
networks,
rather
than
hierarchical
structures,
are
key
elements
in
societal
change,
and
put
a
strong
focus
on
the
processes
that
will
lead
to
change.
Jones
(2011)
has
suggested
another
set
of
tools
to
explain
complex
dynamic
phenomena
in
his
paper
“Taking
Responsibility
for
Complexity
–
How
Implementation
Can
Achieve
Results
in
the
Face
of
Complex
Problems”.
He
promotes
a
greater
recognition
to
general
knowledge
creation
in
development
cooperation;
a
variety
of
sources
of
knowledge
are
needed
to
best
form
actions
in
a
complex
environment.
6.2. Substantialist versus relational thinking
Rosalind
Eyben,
a
researcher
at
IDS
and
founder
of
the
results
initiative
Big
Push
Forward
has
written
extensively
on
the
issue
of
result-‐based
management
in
development
cooperation.
She
argues
that
the
world
can
either
be
seen
from
a
substantialist
or
a
relationist
perspective,
and
that
this
has
consequences
for
how
results
are
perceived.
She
recognises
that
the
present
vogue
of
“managing
for
development
results”
is
an
expression
of
a
historically
dominant
mode
of
thought
in
international
aid
–
substantialism
–
which
sees
the
world
primarily
in
terms
of
‘entities’
such
as
‘poverty’,
‘basic
needs’,
‘rights’,
‘women’
or
‘results’.
In
“Hiding
relations:
The
Irony
of
Effective
Aid”,
Eyben
discusses
that
substantialism
probably
continues
to
survive
due
to
the
perceived
requirements
that
justify
aid
expenditures
in
terms
of
tackling
problems
that
have
been
identified
as
soluble.
She
states
that
“It
would
be
difficult
to
win
an
election
on
the
basis
that
policymaking
is
terribly
messy
and
that
politicians
and
civil
servants
have
very
little
control
over
what
happens”(2011).
She
argues
that
current
approaches
to
managing-‐for-‐results
assume
that
we
are
in
control,
and
that
change
is
predictable.
She
contends
that
the
only
difference
between
the
current
RBM
or
MfDR
approach
from
the
LFA
is
“no
more
than
a
change
from
one
substantialist
discourse
-‐
‘inputs’
-‐
to
another
-‐
‘outputs’”.
She
therefore
concludes
that
the
current
RBM
approach
in
development
cooperation
contains,
and
will
encounter
the
same
difficulties
as
the
LFA-‐approach.

Another
way
of
looking
at
the
world
and
development
cooperation
is
from
a
relationalist
perspective
–
i.e.
placing
more
value
in
relations
and
dialogue,
and
recognising
diverse
perspectives
and
voices
when
accounting
for
more
effective
aid.
Eyben
argues
that
it
is
essential
for
development
cooperation
practitioners
to
also
view,
and
adopt
a
work
method,
from
a
relationist
perspective.
This
must
be
recognised
as
central
to
aid
relationships,
whereby
power
influences
which
ideas
count
and
what
is
deemed
a
‘result’.
If
this
is
done,
a
dialogue
can
commence
based
on
the
recognition
that
effective
aid
means
recognising
diverse
perspectives
and
voices.
In
practice,
she
therefore
argues
that
a
donor
action
should
consist
of
relations
with
selected
recipient
organisations
-‐
those
pursuing
a
social
change
agenda
compatible
with
the
donors’
own
values
and
mission.
This
view
implies
that
the
recipient
organisation
should
not
be
viewed
as
an
instrument,
but
that
support
should
rather
be
provided
to
the
organisations
own
efforts
to
make
change.
In
the
article
Eyben
recognises
that
relational
approaches
do
take
place
in
practice,
but
they
are
officially
ignored.
She
therefore
promotes
further
support
to
practitioners
interested
in
a
relational
approach
from
top-‐management,
and
networks
such
as
the
Big
Push
Forward.
31
6.3. Social change theory
Social
change
theory
emphasises
that
more
focus
in
development
analysis
should
be
put
on
the
power
relations
in
society
and
process.
The
theory
also
has
an
underlying
understanding
for
the
need
to
re-‐distribute
power.
According
to
Guijt
(2010)
pro-‐poor
social
change
efforts
require
conscious
action
and
social
change;
it
is
a
collective
process
of
conscious
efforts
to
reduce
poverty
and
oppression
by
changing
underlying
unequal
power
relationships.
The
theory
seems
to
imply
that
improvements,
for
example
in
development
projects
and
programmes,
are
driven
by
information
and
experience-‐based
reflections,
and
that
assessment
and
learning
are
the
processes
of
ongoing
reflection
about
visions,
strategies,
and
actions
that
enable
continual
readjustment.
According
to
Guijt,
the
mainstream
monitoring
and
evaluation
approaches
(M&E)
do
not
serve
the
type
of
change
processes
discussed
here
as
appropriate
assessment,
learning,
and
processing
requires
adapting
a
combination
of
frameworks,
concepts
and
methods
in
order
to
ensure
that
they
address
information
and
reflection
needs,
and
match
existing
capacities.
However,
this
does
not
imply
that
methods
need
to
be
either
comprehensive
or
complex.
6.4. Cultural theory
Curtis
(2004)
addresses
a
consultant’s
perspective
on
aid
management
in
his
article
“How
We
Think
They
Think:
Thought
Styles
in
the
Management
of
International
Aid”.
The
article
explores
the
idea
that
problems
in
aid
administration
lie
in
the
thinking
processes
that
emerge
within
policy
circles,
at
delivery
levels
within
aid
administration,
and
at
the
interface
between
donor
and
recipient.
In
his
article,
Curtis
applies
the
ideas
of
cultural
theory
to
aid
management
by
stating
that
the
difficulties
in
aid
management
are
caused
by
incompatible
thought
styles
applied
to
the
policy
process
at
different
organisational
levels,
and
that
“the
practical
outcome
is
particularly
problematic
when
parties
at
the
delivery
end
of
aid
seek
to
achieve
complex
outcomes
from
simple,
linear
projects
and
programs—a
habit
to
which
some
donors
seem
to
have
returned,
after
some
years
of
experimentation
with
a
more
flexible
and
responsive
process
of
project
methodology.”

Curtis
states
that
it
is
difficult
to
reconcile
the
theories
of
making
current
aid
management
practices
more
effective
while
donors
seek
to
“influence”
policies
of
the
partners.
In
addition,
policy
influence
is
difficult
because
aid
is
beset
by
misaligned
ends
and
means
–
fundamentally
based
in
unequal
power
relationships
between
donors
and
partners.
The
article
is
critical
of
the
linear
model
proposed
by
results
management,
and
refers
to
the
origins
of
linear
models
as
previously
utilised
in
manufacturing
and
military
campaign
planning.
He
argues
that
the
linear
model,
as
such,
is
counterproductive
to
learning,
and
that
they
are
opposing
principles
of
partnership
and
relationships
in
development
cooperation.
He
therefore
suggests
that
any
effective
intervention
would
have
to
recognise
and
respond
to
the
more
complex
relationships
that
underlie
performance
–patterns
of
personal
influence,
the
power
of
cadre
interests
and
interactions
at
the
administrative/political
interface.
32
6.5. Management theory difficult – but there is no other option⋯
Some
research
admits
that
result
management,
in
its
current
practice,
has
failed,
but
that
there
is
no
other
option
than
to
learn
from
mistakes
and
continue
trying.
Hatton
and
Schroeder
(2007),
in
their
article
“Results-‐Based
management:
friend
or
foe?”
provide
an
evaluator’s
perspective
on
RBM.
In
the
article
they
agree
that
the
introduction
of
RBM
in
development
cooperation
has
indeed
been
challenging
and
complex.
However
they
question
whether
if
RBM
is
an
enemy
that
detracts
from
development
work.
Based
on
their
personal
experience,
they
claim
that
“The
agencies,
organizations
and
corporations
that
embrace
RBM
typically
produce
strong
results”.
They
therefore
argue
that
organisations
should
adapt
and
implement
RBM
more
aggressively,
meaning
that
there
need
to
be
consequences,
such
as
losing
the
project
if
reporting
is
not
based
on
the
RBM
framework,
if
developing
country
partners
do
not
participate
in
all
facets
of
project
development
and
implementation,
etc.
Robinson
(2007),
a
fellow
of
Drucker,
concludes
in
his
article
“The
Challenge
of
Tyranny”
that
blame
for
current
events
-‐
with
a
breakdown
of
social
and
political
structure
-‐
is
attributable
to
the
practices
of
the
management
profession.
However,
he
states
that
there
is
no
other
option
than
to
work
to
improve
the
current
management
practices.
He
states
that
Drucker’s
major
contribution
to
management
was
that
people
always
come
first,
and
that
management
is
about
people
and
their
relationships
with
each
other.
The
need
to
organise
is
inherent
to
human
nature.
He
has
found
a
positive
benefit
in
offering
opportunities
for
enhancement
in
lower
level
management
training
to
organisations.
His
contribution
to
the
debate
is
therefore
that
there
is
an
urgent
need
to
promote
improved
coordination
of
management
educators
and
trainers,
based
on
improved
analysis
of
training
needs
in
relation
to
development
aims.
In
a
presentation
made
by
Barder
(2011)24
with
the
title
“SIMPLIFICATION
&
RESULTS:
How
a
focus
on
results
could
increase
rather
than
reduce
country
ownership?”
reference
is
made
to
the
problems
with
results-‐based
aid
when
used
incorrectly
–
i.e.
that
there
is
not
enough
scope
for
learning
when
donors
determine
funding
requirements
in
advance,
that
a
results
bureaucracy
is
created
if
separate
project
and
programme
systems
are
created,
and
that
there
is
a
reduced
scope
for
pooling
and
collaboration
if
donors
want
to
demonstrate
the
results
of
their
own
aid
etc.
However,
he
then
asks
24
This
presentation
was
made
at
a
Conference
organised
by
SITE
and
MFA
in
Sweden
on
30th
May
2011,
with
the
title
“Challenges
for
Aid
Effectiveness”.

the
question
“Is
result
based
aid
really
the
bad
guy?”
and
states
that
results
measurement
is
hard
due
to
imperfect
information,
complexity
of
systems,
and
contestable
goals,
but
that
it
is
necessary
due
to
fiduciary
duties,
political
pressures
and
a
desire
to
improve.
He
states
that
“Simplification”
is
probably
the
missing
principle
of
aid
effectiveness,
and
that
a
link
to
results
is
most
likely
necessary,
if
not
necessarily
sufficient
to
bring
it
about.
33
6.6 Some conclusions
The
differences
between
the
theoretical
approaches
may
not
appear
to
be
as
large.
They
all
argue
for
more
focus
on
analysis
of
how
change
occurs,
and
that
improved
decision-‐making
will
require
a
mind-‐set
shift.
Furthermore,
they
all
insist
that
results-‐based
management
has
never
truly
been
implemented
as
intended,
and
they
therefore
suggest
a
number
of
methods
and
tools
to
this.
However,
these
similarities
in
analysis
and
concepts
hide
the
fact
that
there
are
epistemological
and
ontological
differences
between
the
theories.
Mainstream
results-‐based
management
is
based
on
an
assumption
that
change
occurs
in
a
linear
fashion
where
a
set
of
activities
results
in
outputs,
outcomes
and
ultimately
impact.
To
achieve
the
best
possible
results,
it
is
therefore
important
to
have
a
solid
theory
of
change
that
outlines
the
links
in
the
results
chain
from
inputs
to
impact.
Such
theories
of
change
should
ideally
be
developed
through
empirically
testing
hypotheses
for
change.
This
can
be
done
via
different
methods
such
as
monitoring
by
objective
quantitative
indicators,
theory-‐based
evaluations
and
rigorous
impact
evaluations.
In
contrast,
the
complexity/social
change/cultural
theories
do
not
regard
change
as
a
linear
process
where
certain
interventions
and
“best
practices”
lead
to
predictable
results.
Instead,
change
is
seen
as
a
dynamic
and
complex
process
where
interventions
and
practices
are
shaped
by
contextual
factors
and
human
relationships.
From
this
perspective,
a
results
analysis
needs
to
take
into
account
a
number
of
factors,
including
some
that
are
based
on
perceptions
rather
than
objectively
verifiable
quantitative
indicators.
Here,
approaches
such
as
outcome
mapping,
that
focuses
on
assessing
changes
in
the
behaviour
of
the
people
with
whom
a
development
initiatives
works
most
closely,
and
utilisation-‐focused
evaluation
as
proposed
by
Michael
Patton
(2008),
that
integrates
evaluation
in
decision
making
processes,
are
appropriate
methods
for
analysis
and
informed
decision
making.
Hence,
the
theoretical
perspective
shapes
the
way
change
is
understood
(linear
or
complex),
how
projects
and
programmes
should
be
designed
(based
on
“best
practices”
of
“what
works”
or
on
a
contextual
analysis
involving
the
change
agents)
and
thereby
methods
for
evaluating
and
managing
for
results.
The
term
“results-‐based
management”
is
commonly
used
in
development
cooperation
for
management
practices
that
are
based
on
linear
theories
of
change,
and
associated
with
monitoring
and
evaluation
practices
that
emphasise
quantitative
data.
However,
as
we
pointed
out
in
chapter
4,
when
this
term
came
into
fashion
in
the
1990’s
it
was
to
be
regarded
in
development
cooperation
as
a
return
to
the
notions
of
Drucker,
and
his
insistence
on
successful
results
orientation
as
a
mind-‐set
and
a
perspective
on
management
and
learning,
rather
than
a
precise
set
of
instructions.
From
that
perspective,
concepts
such
as
results-‐based
management
or
results
management
could
be
used
for
a
variety
of
practises,
including
approaches
that
are
underpinned
by
complexity
theory
or
social
change
theory.

The
fact
that
the
term
“results-‐based
management”
can
be
used
for
practices
underpinned
by
very
different
ontological
and
epistemological
assumptions
can
cause
confusion,
not
least
when
it
comes
to
the
practical
implementation
of
results-‐based
management.25
Nonetheless,
we
have
chosen
to
apply
the
term
in
a
very
wide
sense
in
this
study,
thus
implying
that
we
do
not
regard
it
as
a
concept
that
describes
exclusively
mainstream
results
management
practices.
34
25
It
is
also
worth
pointing
out
that
different
theoretical
perspectives
are
driving
and
underpinning
some
of
the
most
contentious
current
debates
on
results,
for
example
the
debate
on
the
value
of
rigorous
impact
evaluations.

35
7. Current international initiatives in results management
So
far,
our
overview
has
helped
us
to
find
different
theoretical
explanations,
and
some
answers,
to
the
challenges
of
result
based
management.
Similarly,
the
raison
d’être
for
different
international
initiatives
related
to
RBM
could
help
us
to
understand
and
explain
the
need
for
support
mechanisms,
the
recent
push
for
the
agenda,
and
the
emerging
global
debate
on
the
issue.
We
have
chosen
to
group
the
initiatives
into
the
following
categories:
1. Result
Based
Financing
Initiatives
–
Initiatives
that
focus
on
financial
incentives
for
the
achievement
of
results
2. Initiatives
that
support
developing
countries
3. Initiatives
that
support
donors
4. Global
Measurement
Initiatives
5. Initiatives
questioning
the
results
agenda
We
will
focus
on
how
the
initiatives
define
their
own
“raison
d’être”.
7.1. Results Based Financing – Initiatives that focus on financial incentives for
the achievement of results
Results
Based
Financing
(RBF)
initiatives
are
important
from
a
result
management
perspective
because
they
focus
on
results
management
in
donor-‐recipient
relations.
We
have
analyzed
some
of
the
literature
on
RBF
initiatives
in
this
review
due
to
a
currently
hot
debate
on
finding
the
most
effective
financing
forms
in
development
cooperation.
During
the
last
ten
years,
several
initiatives,
focusing
on
using
different
forms
of
incentives
to
achieve
objectives,
have
received
increased
attention
from
bilateral
donors.
Result
Based
Financing
(RBF)
initiatives
is
the
generic
term
used
for
strategies
that
focus
on
a
results
orientation
of
the
financing
instruments
in
aid.
Some
agencies
also
use
the
term
Result
Based
Aid
(RBA).26
Today,
more
funding
is
provided
through
these
types
of
approaches,
and
they
are
increasingly
used
to
support
a
whole
sector/government
plan.
The
rationale
for
many
of
these
approaches
is
that
aid
funding
is
conditioned
to
financing
outputs
or
outcomes
rather
than
inputs
and
processes.
Such
an
approach
is
assumed
to
improve
accountability
to
beneficiaries,
and
create
incentives
for
efficient
delivery.
In
practice,
this
means
that
you
define
beforehand
exactly
what
you
will
pay
for,
and
only
release
payments
if
these
results
are
achieved;
failure
to
deliver
results
has
an
immediate
effect
on
funding
–
often
irrespective
of
the
reasons
for
26 The
World
Bank,
for
example,
suggests
that
results
based
aid
involves
a
donor
as
the
principal,
and
a
national
government
as
the
agent,
while
results
based
financing
involves
a
national
or
sub-‐national
government
body
(e.g.
a
Ministry
of
Health,
a
district
health
authority)
as
the
principal
and
a
range
of
possible
agents
(individuals,
NGOs,
sub-‐national
government).
DFID
uses
a
rather
different
approach
in
which
RBA
and
RBF
are
distinguished
according
to
their
funding
sources
and
the
contracting
arrangements
used.
Approaches
involving
donor
funding
–
and
where
the
contract
is
between
a
donor
and
national
government
–
would
be
classified
as
results
based
aid.
Approaches
using
domestic
funds
and
where
the
contract
is
between
government
or
sub-‐national
entity
(whether
government
or
NGO)
and
the
implementing
partner
(which
could
be
a
sub
national
level
of
government,
an
NGO
or
even
an
individual)
would
be
classified
as
results
based
financing.

36
the
failure
to
achieve
the
result
(Pearson).
However,
some
of
the
approaches,
such
as
Poverty
Reduction
Budget
Support
(PRBS)
and
Millennium
Challenge
Account
(MCA),
do
not
enforce
an
immediate
connection
between
financing
and
results.
In
for
example
PRBS
most
donors
conduct
analysis
based
on
performance.
This
analysis
is
then
used
in
dialogue,
and
funding
is
decided
after
that.
Many
of
the
approaches
claim
to
reduce
transactions
costs,
reduce
the
impact
of
information
asymmetries,
and
promote
ownership
by
developing
countries
as
the
approaches
are
more
“hands-‐
off”
than
traditional
input-‐based
aid.
Some
of
the
current
initiatives/financing
forms/schemes
are:
Initiative/financing
form/scheme
Short
description
Global
Program
for
Output
Based
Aid
(GPOBA)
Multi-‐donor
partnership
and
trust
fund
established
to:
i)
fund
and
facilitate
the
preparation
of
OBA
projects
in
which
payment
is
made
to
an
implementing
agent
(usually
private
sector
but
potentially
non
governmental
organisations,
and
usually
in
the
utilities
sector)
for
each
unit
of
output
supplied;
and
ii)
document
and
disseminate
lessons
learned.
Cash
On
Delivery
Aid
(CODA)
A
concept
for
making
payments
to
government
in
return
for
achievement
of
specific
results
(e.g.
increase
in
primary
school
enrolment).
Vouchers
Reimbursement
made
to
accredited
providers
on
the
basis
of
services
delivered
to
voucher
recipients.
Conditional
Cash
Transfers
Payment
made
to
targeted
beneficiary
in
return
for
using
specified
services.
Heavily
focused
in
middle
income
countries
using
domestic
funds.
Poverty
Reduction
Budget
Support
(PRBS)
Payment
made
to
government
in
return
for
commitment
to
good
governance
and
satisfactory
progress
in
poverty
reduction.
Variable
or
performance
related
tranche
payments
are
a
form
of
results
based
aid.
European
Commission
MDG
Contracts
Payment
made
to
government
in
return
for
commitment
to
good
governance
and
satisfactory
progress
in
poverty
reduction.
Variable
tranche
is
a
form
of
RBA.
Schemes
have
been
established
but
are
new.
Millennium
Challenge
Account
(MCA)
Payment
made
to
government
in
return
for
demonstrable
commitment
to
democracy,
good
governance,
‘economic
freedom’
and
pro-‐poor
public
services.
From
Pearson
(a
modified
table,
here,
only
with
description
of
the
cross-‐sectoral
initiatives/schemes)

Pearson
differentiates
between
the
approaches
by:
a)
who
provides
the
funds;
b)
who
delivers
the
results;
and
c)
how
performance
is
measured
and
monitored.
Also
Savedoff
(2011)
has
attempted
to
categorise
incentive
schemes
into
two
dimensions:
the
agent
whose
behaviour
the
incentive
seeks
to
change,
and
the
specificity
of
the
output
or
outcome
assessment.
Both
come
to
the
conclusion
that
there
are
a
vast
number
of
global
initiatives
that
only
seek
to
address
a
few
results
(for
e.g.
the
Roll
Back
Malaria
initiative
that
only
measures
outcomes
and
impacts
related
to
malaria)
but
that
many
of
these
initiatives
are
not
based
on
the
principles
of
aid
effectiveness,
do
not
use
country
systems,
and
are
not
aligned
to
the
country
strategies
and
priorities.
What
is
then
different
between
RBF
and
traditional
aid?
An
answer
to
this
question
requires
an
analysis
of
the
results
and
conditions
set
in
the
individual
aid
contract.
Obviously,
results,
in
some
form,
are
part
of
all
aid
contracts
–
however,
the
main
difference
is
that
the
RBF
link
between
results
and
payment
is
more
direct.
Pearson
(2010)
states
that
the
method
is
not
so
different
from
other
conditional
aid
mechanisms
where
the
donor
sets
out
what
it
wants
to
be
delivered,
and
what
it
is
willing
to
pay
for
it.
However,
he
claims
that
the
style
of
delivering
aid
differs
between
aid
as
emphasised
in
the
Paris
Principles
and
RBF:
In
Paris-‐style
aid
(such
as
budget
support)
donors
are
more
interested
in
rewarding
commitment
to
achieve
results
than
the
actual
delivery
of
results,
which
allows
them
to
continue
to
provide
support
when
uncontrollable
factors
constrain
result
achievement.
He
means
that
a
difference
also
lies
in
the
interests
of
promoting
predictability.
In
Paris-‐style
aid,
the
correct
response
to
a
failure
to
deliver
results
through
lack
of
commitment
would
be
to
reduce
support
in
future
periods,
not
the
current
period.
He
therefore
concludes
that
both
aid-‐
styles
try
to
deliver
results
–
yet
they
do
so
in
a
different
manner.
The
motives
and
arguments
for
the
introduction
of
different
RBF
initiatives
seem
quite
similar.
They
base
their
arguments
on
the
high
degree
of
fragmentation,
multitude
of
uncoordinated
development
assistance
providers,
high
transaction
costs
incurred
by
recipient
countries
in
their
engagements
with
the
development
assistance
providers,
limited
ownership
of
partner
countries,
the
lack
of
accountability
between
all
involved
stakeholders,
and
lack
of
transparency.
A
recent
initiative,
TrAiD+
mechanism27,
defined
in
a
Working
Paper
from
the
Centre
for
Global
Development
(2011)
attempts
to
address
all
key
problems
simultaneously.
One
interesting
aspect
of
TrAid+
is
that
it
tries
to
go
further
with
the
results
agenda,
and
create
a
“market
of
global
results”
where
investors
could
choose
what
type
of
projects
to
fund,
based
on
results
achieved.
In
their
paper
they
ask
“if
investors
can
be
found
for
the
future
delivery
of
commodities,
or
for
financial
indices
based
on
stock
market
performance,
why
not
find
investors
interested
in
the
delivery
of
children
completing
primary
school,
or
surviving
to
age
five?”
Although
not
yet
tried
out
in
any
country
or
programme,
the
“Cash
on
Delivery”
(COD-‐Aid)
initiative
has
generated
much
discussion
in
the
media,
predominantly
in
the
US
and
the
UK28,
as
well
as
in
the
greater
aid
community
more
recently.
The
initiative
has
received
official
support
from
Dfid,
Sida,
the
27
The
mechanism
aims
to
track
results,
improve
coordination
between
actors,
and
improve
transparency,
accountability,
and
country
ownership
by
creating
a
market
for
certified
development
outputs.
The
initiative
suggests
that
separate
TrAid+
Governance
bodies
be
set
up
in
development
countries
to
act
as
a
third-‐party
stamp
of
approval
to
donors,
tax
payers,
recipient-‐country
governments,
service
providers,
and
beneficiaries.
The
basic
idea
is
that
a
service
provider
(e.g.
an
NGO),
willing
to
provide
services
for
e.g.
extension
of
clean
water
would
approach
the
TrAid+
platform,
and
prepare
an
intervention
document
according
to
TrAid+
Guidelines.
Once
the
intervention
has
been
approved
by
the
TrAid+
Governance
board
the
intervention
would
be
accessible
online,
making
it
easier
for
funders
interested
in
purchasing
certified
development
outputs.
28
See
for
ex.
Eyben
(2011),
Melameb
(2011),
Rosenberg
(2011),
37

Gates
Foundation,
and
the
UK
State
Secretary
Andrew
Mitchell29.
The
initiative
has
placed
the
results
focus
as
a
top
priority
in
the
recent
debate
on
development
cooperation.
38
The
COD-‐aid
initiative
has
encountered
a
lot
of
support,
but
also
criticism
and
questions
regarding
data
collection
concerns
in
partner
countries,
faulty
indicators,
the
risk
of
no
financing
received
if
outcomes
and
results
are
not
achieved,
and
the
risks
of
manipulation
of
data,
risks
of
promoting
short
term
results
and
ignore
sustainability
etc.
Rosenberg
(2011)
comments:
Data
collection
is
an
administrative
burden
for
a
government.
But
let’s
compare
it
to
what
foreign
aid
programs
today
demand
from
poor-‐country
governments:
endless
paperwork
to
satisfy
the
requirements
of
multiple
donors
that
their
money
is
being
well-‐spent.
The
vast
majority
of
these
bureaucratic
requirements
bring
no
benefit
to
the
poor
country
at
all
beyond
that
of
keeping
donors
happy.
Cash
on
delivery,
by
contrast,
would
dispense
with
all
that.
Instead,
it
would
require
governments
to
measure
whether
programs
are
working.
This
is
a
burden
with
tremendous
benefits
for
a
poor
country.
The
COD-‐aid
initiative
really
merits
attention
for
its
hands-‐off
approach,
and
the
intention
to
simplify
control
aspects
regarding
e.g.
financial
management.
But
is
it
feasible?
It
would
imply
that
donors
relinquish
control
of
budgets,
accounting,
and
corruption
aspects,
which
might
not
be
allowed
under
the
donors’
laws
and
regulations
regarding
financial
management
and
control.
Hence
there
is
a
risk
that
more
outcome
control
would
only
add
to
the
extensive
input
and
output
control
already
taking
place.
Furthermore,
given
that
partner
countries
have
very
weak
systems
for
monitoring
and
evaluation
one
should
not
underestimate
the
costs
of
setting
up
data
information
and
validity
systems.
And
even
if
investments
were
made
in
such
systems,
the
well
known
problems
of
finding
valid
indicators,
which
could
be
monitored
in
a
reliable
manner,
would
persist.
As
mentioned
above
there
are
a
number
of
other
questions
raised
in
regard
to
COD-‐aid.
For
example:
What
happens
when
targets
are
not
met?
Will
the
countries
not
receive
any
financing
at
all?
Is
there
a
risk
that
the
initiatives
will
only
finance
improvement
of
information
systems,
and
third
party
control
over
the
figures?
Will
short-‐term
results
be
rewarded,
whereas
long-‐term
sustainable
developmental
changes
are
disregarded?
As
experience
form
the
World
Bank
indicates30
concerns
of
this
type
may
lead
donors
to
focus
on
results
of
which
the
implementing
partner
have
sufficient
control,
and
which
are
easy
to
measure
and
validate.
Such
an
approach
would
re-‐introduce
all
the
implementation
problems
which
are
well
known
in
RBM,
and
which
we
have
accounted
for
earlier
in
this
paper:
i.e.
focus
on
what
is
measurable
rather
than
what
is
relevant
which
usually
leads
to
output-‐orientation
rather
than
outcome-‐orientation.
Whether
the
RBF
initiatives
will
have
a
large
effect
on
aid
remains
to
be
seen.
Rogerson
(2011)
from
the
OECD/DAC
states
that
measurement,
attribution,
and
perverse
incentive
problems
generally
arise
with
many
target-‐driven
schemes,
and
that
tight
earmarking
of
aid
runs
counter
to
local
ownership
of
spending
choices
and
channels.
He
does,
however,
praise
the
COD-‐aid
initiative
because
its
design
has
tried
to
take
account
of
these
difficulties.
Others
such
as,
Glennie
(2011)
critically
state
that
RBF
will
be
no
different
than
other
modalities
in
history
because
donors
simply
won’t
accept
a
hands-‐off
approach,
and
that
donors
incentives
are
not
in
line
with
developmentally
29
http://www.dfid.gov.uk/Media-‐Room/Speeches-‐and-‐articles/2010/Placing-‐women-‐at-‐the-‐heart-‐of-‐development/
30
As
reported
at
a
session
on
the
topic
of
results
based
aid
at
the
High
Level
Forum
for
Aid
Effectiveness
in
Busan.

sound
decision
making.
Savedoff
(2011)
takes
a
more
careful
approach
and
proposes
that
recent
approaches
are
positive
in
that
they
focus
on
incentives
and
how
people,
organisations,
and
countries
respond
to
incentives
-‐
which
are
important
in
complex
environments,
such
as
development
cooperation.
However,
he
states
we
should
not
fool
ourselves
into
believing
that
there
are
perfect
designs.
To
date,
there
seem
to
be
very
few
evaluations
on
the
effectiveness
of
RBF
approaches.
A
review
commissioned
by
Norad
(2011)
concluded
that
financial
incentives
seem
to
be
effective
in
the
short
term
for
simple
and
clearly
determined
behaviour
objectives.
There
is
less
experience
as
to
whether
financial
incentives
can
provide
longer-‐term
changes.
The
report
acknowledges
some
positive
room
for
potential,
such
as
increasing
the
overall
focus
on
problem
solving,
attention
to
results
in
general,
improving
motivation,
etc.
But
it
also
acknowledges
negative
effects
such
as
RBF
contribution
to
unintended
behaviour,
gaming,
cherry
picking
(choosing
patient
groups
that
make
it
easier
to
achieve
the
targets),
etc.
Pearson
(2011)
concludes
that
there
is
a
need
for
a
positive
but
cautious
approach
towards
RBF
initiatives.
He
finds
good
design
to
be
essential.
The
approach
needs
to
be
tailored
to
local
circumstances,
and
may
be
better
suited
to
some
settings
than
others;
results-‐based
approaches
are
unlikely
to
be
sufficient
to
deliver
desired
results
-‐
complementary
actions
will
usually
be
required.
He
also
concludes
that
the
case
for
general
budget
support
remains
strong.
He
states
that:
If
things
are
not
working
perfectly
there
is
often
a
rush
to
find
new
approaches,
but
it
is
important
not
to
overlook
existing
ones.
Although
not
universally
applicable,
general
budget
support
is
a
form
of
results
based
aid
which
is
results
focused,
tried
and
tested,
has
a
sound
record
of
achievement
(although
it
has
not
always
been
applied
to
maximum
effect
and
lessons
not
always
applied)
and
is
fully
aligned
with
national
systems.
This
is
more
than
can
be
said
for
many
of
the
newer
approaches.
This
is
also
the
conclusion
we
would
like
to
draw
in
our
review.
Many
of
the
new
initiatives
are
designed
to
work
on
a
project
basis,
and
seem
to
disregard
the
complexity
of
results
management
as
such.
There
is
seldom
only
one
solution
on
how
to
achieve
the
best
results.
When
it
comes
to
results
of
development
cooperation
it
is
also
important
to
reiterate
that
the
principles
outlined
in
the
Paris
Declaration
were
based
on
decades
of
hard
learned
lessons
of
what
works
and
what
doesn’t
work.
As
The
Evaluation
of
the
Paris
Declaration
(2011)
concludes
the
principles
remain
relevant
and
useful.
The
main
problem,
according
to
the
evaluation,
is
thus
not
that
we
have
the
wrong
methods
for
delivering
effective
aid
to
improve
development
outcomes,
but
that
donors
in
particular
do
not
adhere
to
the
agreed
principles
and
use
the
methods
recommended.
However,
the
evaluation
also
concludes
that
more
work
needs
to
be
done
regarding
results
and
accountability
mechanisms
in
particular.
39
7.2. Initiatives that support partner countries
During
the
latest
decade
several
support
initiatives
focusing
on
improving
partner
countries
results
management
systems
have
been
taken
under
the
OECD/DAC’s
Joint
Venture
for
Managing
for

Development
Results
and
by
the
Independent
Evaluation
Group
of
the
World
Bank
(IEG)31.
These
initiatives
have
developed
a
number
of
support
manuals
on
how
to
improve
Results
Management
practices
in
developing
countries,
such
as
“Ten
Steps
to
Result
Based
Management”
by
the
World
Bank,
“How
to
build
M&E
systems
to
build
better
Governance”
by
the
IEG,
and
the
“Sourcebook
on
Emerging
Good
Practice
in
Managing
for
Development
Results”
by
the
OECD/DAC.
Specific
Manuals
have
also
been
developed
within
different
sectors,
such
as
“Tracking
results
in
agriculture
and
rural
development
–
A
sourcebook”
by
FAO
and
the
World
Bank.
All
of
the
support
manuals
provide
examples
of
implementation
of
result
based
management,
and
tools
and
lessons
for
how
to
improve
the
practice.
As
part
of
the
Joint
Venture
on
Managing
for
Development
Results
(later
Global
Partnership
for
MfDR),
regional
Communities
of
Practice
have
been
set
up
in
Asia,
Latin
America
and
Africa.
Their
aim
is
to
reinforce
the
capacity
to
manage
for
development
results
through
experience
sharing
and
peer
learning.
They
provide
a
forum
to
exchange
solutions
that
increase
evidence-‐based
decision
and
policy
making
for
effective
delivery
of
development
results.
Each
of
them
is
supported
through
specific
mechanisms,
and
develops
different
activities.
Each
one
of
the
Communities
of
Practices
has
developed
their
own
Sourcebooks.
The
web-‐pages
of
the
different
support
initiatives
contain
support
documentation
on
how
to
improve
results
management
in
partner
country
contexts.
It
is
difficult
to
find
any
external
evaluations
or
documented
research
on
the
different
programmes.
The
IEG-‐webpage
contains
some
evaluations
on
implementation
of
Monitoring
and
Evaluation
in
countries
such
as
Colombia,
South
Africa,
Canada,
and
the
U.S.
40
7.3. Initiatives that support donors
On
the
donor
side,
similar
global
initiatives
to
support
donors
in
their
results
management
do
not
exist.
It
seems
as
if
reforms
in
donor
agencies
and
international
agencies
generally
take
place
without
a
lot
of
exchange
of
experiences,
probably
because
results
management
reforms
usually
are
driven
within
the
civil
service
of
the
particular
countries.
However,
the
OECD/DAC
has
reviewed
and
provided
some
guidance
that
support
donor
agency
reform:
-‐
In
2000,
a
comparative
review
on
donor
performance
in
RBM
was
commissioned
by
the
DAC
Evaluation
Network32.
Many
of
the
outcomes
are
summarised
in
chapter
3.
31
IEG
provides
technical
assistance
to
member
developing
countries
for
designing
and
implementing
effective
monitoring
and
evaluation
(M&E)
systems,
and
for
strengthening
government
evaluation
capacities
as
an
important
part
of
sound
governance.
IEG
prepares
resource
materials,
with
case
studies
demonstrating
good
or
promising
practices,
which
other
countries
can
refer
to
or
adapt
to
suit
their
own
particular
circumstances.
World
Bank
support
to
strengthen
M&E
systems
in
different
countries
has
grown
substantially
in
the
past
decade.
Furthermore
IEG,
supported
by
many
bilateral
donors,
has
led
the
International
Program
for
Development
Evaluation
Training,
aimed
at
individual
training
in
monitoring
and
evaluation,
for
many
years.
They
have
also
has
taken
the
initiative
for
a
program
aimed
at
setting
up
a
program
for
longer
term
capacity
building
(Regional
Centers
for
Learning
on
Evaluation
and
Results
(CLEAR))
in
Africa,
Asia
and
Latin
America.
Many
bi-‐laterals
support
these
programmes
financially.
32
Conducted
by
Anette
Binnedjikt.

-‐
RBM
has
been
brought
up
in
DAC
Peer
Reviews
of
some
DAC
member
countries
(although
as
one
of
many
aspects
brought
up
in
the
Peer
Reviews).
A
summary
of
findings
from
the
DAC
Peer
Reviews
was
published
in
2011.
Some
of
the
outcomes
are
summarised
in
chapter
3.
41
-‐
The
OECD/DAC
has
produced
one
guidance
document
on
aid
management
in
general,
The
“Managing
Aid”
report.
The
report
provides
11
lessons
on
aid
management;
one
of
them
on
result
management
which
states
that
“Develop
a
stronger
culture
of
managing
for
results
and
align
incentives
accordingly,
but
to
promote,
not
weaken,
local
structures
of
accountability”.
-‐
As
part
of
the
preparations
for
the
High
Level
Forum
in
Busan,
the
Global
Partnership
for
Results
has
produced
a
“Good
Practice
Principles
on
Result
Reporting
in
donor
agencies”.
The
paper
proposes
a
set
of
principles
for
reporting
development
results,
as
applicable
to
donors
and
other
stakeholders.
-‐
The
Managing
for
Development
Results
Group
has
produced
a
guide
on
“Incentives
for
aid
effectiveness
in
donor
agencies
–
Good
Practice
and
Self-‐Assessment
tool”.
In
sum,
it
appears
that
there
are
many
global
initiatives
and
support
mechanisms
in
place
to
support
partner
countries
efforts
with
result
based
management.
There
also
exist
also
supporting
guidance
documents
for
donors.
However,
there
seem
to
be
a
lack
of
practical
guidance
for
donors
on
how
to
actually
set
up
their
result
management
systems
to
best
adhere
to
development
countries
systems.
7.4. Global measurement initiatives
Several
global
measurement
initiatives
has
arisen
in
response
to
the
international
WHAT-‐goals
–
i.e.
the
Millennium
Development
Goals
and
other
global
goals
in
different
sectors
(such
as
Education
For
All
goals)
-‐
as
well
as
goals
for
aid
disbursement
(e.g.
the
0,7%
goal).
Several
Quality
Assurance
mechanisms
have
arisen
in
response
to
the
HOW
goals,
and
the
Paris
Declaration
to
measure
donor
performance,
management
and
policy
practices.
These
include:
-‐
Paris
Monitoring
Survey,
by
OECD/DAC
-‐
Commitment
to
Development
Index,
by
Centre
for
Global
Development
-‐
Aid
Watch,
by
Development
Today
-‐
Quality
of
ODA,
by
Brookings
Institute
-‐
Transparency
Index,
by
Brookings
Institute
-‐
Aid
Quality
and
Donor
Rankings,
by
the
World
Bank
-‐
Forward
Spending
Plans
on
Aid,
by
OECD/DAC
-‐
An
Index
of
Donor
Performance,
by
Roodman
D.
Centre
for
Global
Development
Easterly
(2010)
states
that
we
are
probably
still
in
the
“let
thousand
flowers
bloom”
phase
of
aid
agency
monitoring
literature,
and
discusses
that
this
is
probably
so
because
aid
agencies
lack
the
normal
feedback
loops
available
to
democratic
or
market
actors,
and
thus
cannot
observe
the
dissatisfaction
of
voters
or
customers.
He
states
that
aid
beneficiaries
have
no
vote
and
no
purchase
decisions
by
which
they
could
communicate
dissatisfaction
to
aid
agencies.
He
concludes
that
a
variety
of
independent
academic
checks
on
aid
agencies
are
therefore
useful
in
filling
the
void.
Knack
(2010)
discusses
the
role
of
these
public
rankings,
and
states
that
they
can
inspire
both
bilateral
and
multilateral
donors
to
improve
their
efforts,
effectiveness,
and
efficiency
in
ways
that
will
move
them
up
in
the
rankings.

According
to
Knack
(2010),
who
refers
to
Ramankutty
(2009),
there
is
also
evidence
that
donors
do
in
fact
pay
attention
to
these
rankings
and
care
about
public
perceptions.
However,
Knack
also
notes
that
the
donor
performance
as
measured
in
the
rankings
is
based
on
largely
untested
beliefs
about
best
practices
in
aid
management.
Similarly
Easterly
(2010)
concludes
that
there
are
simply
no
reliable
impact
measures
available
across
agencies
and
that
it
is
simply
not
possible
to
measure
the
impact
of
aid
on
beneficiaries.
Therefore
all
of
the
quality
rankings
have
to
appeal
to
a
common
sense
consensus
that
very
bad
performance
on
the
indicators
would
make
strong
positive
aid
impact
less
likely.
Global
quality
measurement
initiatives
could
provide
the
pressure
to
improve
results
management
in
partner
countries.
One
of
the
Paris
Declaration
indicators
measures
whether
countries
have
results
oriented
frameworks.
However,
there
is
currently
no
method
for
measuring
whether
partner
country
results
frameworks
are
actually
used
by
donors.
Yet
the
Brookings
Institute
has
suggested
an
indicator,
in
their
Quality
of
ODA,
for
the
amount
of
donor
support
that
goes
to
countries
with
qualitative
results-‐oriented
frameworks.
There
are
also
several
tools
that
have
been
developed
by
different
organisations
to
conduct
capacity
analyses
of
the
countries
result
based
management
systems,
such
as
the
joint
World
bank
OECD/DAC
Capacity
Scanning
for
Managing
for
Results
(Cap-‐Scan).
Several
of
the
bilateral
and
multilateral
donors
have
their
own
capacity
measurement
tools.
The
performance
of
partner
country
results
management
systems
are
measured
both
through
the
Paris
Indicators,
by
the
OECD/DAC,
as
well
as
through
different
Public
Expenditure
and
Financial
Accountability
Indicators
(PEFA).
None
of
the
indicators
in
the
rankings
nor
a
specific
capacity
tool
seem
to
have
been
developed
to
measure
aspects
of
the
quality
of
bilateral
donor
result
based
management
systems.
On
the
multilateral
side,
the
Multilateral
Organization
Performance
Assessment
Network
(MOPAN)
measures
the
effectiveness
of
multilateral
organisations,
including
their
quality
of
results-‐based
budgeting.
From
2009
and
onwards,
an
annual
report
on
Multilateral
Aid
Report
has
been
produced
by
the
DAC.
The
report
compares
practices
in
multilateral
aid
performance.
However,
neither
the
reports
nor
the
DAC
Peer
Reviews
put
much
emphasis
on
assessment
of
results
management
of
donors.
42
In
sum,
there
seem
to
be
a
growing
number
of
initiatives
that
measure
and
compare
donor
performance
in
general.
However
there
are
no
instruments
or
tools
that
compare
donor
performance
in
results
management,
or
determine
if
they
actually
use
partner
country
results
management
systems.
Yet
several
tools,
and
some
indicators
in
the
rankings,
assess
and
measure
partner
country
results
management
systems.
7.5. Initiatives questioning the agenda
In
recent
years,
a
few
global
initiatives
have
arisen
as
a
direct
response
to
the
strong
push
for
results.
The
3IE
initiative
arose
due
to
increasing
pressure
to
show
results
from
public
spending.
The
rationale
for
the
initiative
can
be
found
in
the
Centre
for
Global
Development
publication
“When
will
we
ever
learn?”
and
its
Evaluation
Gap
working
group.
In
the
report
it
was
realised
that
there
was
shockingly
little
concrete
evidence
about
what
worked
and
what
didn’t,
and
that
outcome
monitoring
said
nothing
about
how
an
agency’s
programmes
were
affecting
outcomes.
The
mission

43
of
the
initiative
is
to
fund
relevant
policy
impact
evaluations,
provide
technical
support
to
organisations
that
conduct
these
impact
evaluations,
engage
with
policy
makers,
and
produce
different
resources
and
various
publications.
The
Big
Push
Forward
is
a
network,
in
the
UK
with
around
300
subscribed
practitioners
who
identify
and
share
strategies
to
encourage
funders
and
implementers
to
experiment
with
and,
when
appropriate,
adopt
additional,
useful
approaches
to
impact
assessment
and
reporting
of
international
aid
programmes
and
projects.
The
initiative
organises
workshops
and
shares
publications
on
their
web
site.
The
network
organises
thematic
groups
for
further
analysis
on
different
topics
such
as:
reclaiming
the
meaning
of
value
for
money,
challenging
dominant
discourses,
collaborating
with
people
inside
aid
agencies
who
are
dissatisfied
with
the
audit
culture,.
The
initiative
seeks
to
find
constructive
ways
to
conceptual
and
methodological
advancement
for
assessing
the
impact
of
development
aid
in
support
of
a
fairer
world,
beyond
the
narrow
bureaucratic
protocols
that
assume
guaranteed
predictable
outcomes.
There
are
also
quite
a
few
development
blogs
and
virtual
networks
that
discuss
and
question
the
agenda,
such
as
the
OXFAM
blog
“From
Poverty
to
Power,
a
blog
edited
by
Duncan
Green”.
On
the
blog
a
discussion
has
lately
been
ongoing
for
example
on
the
topic
of
the
risks
of
the
Results
agenda
vs.
the
Risk
of
not
having
a
results
agenda.33
In
this
discussion
questions
are
raised
regarding
evidence
on
what
to
do
in
development
aid,
but
that
there
seems
to
be
a
less
intense
focus
on
evaluating
the
successes
of
the
results
agenda,
or
development
cooperation
methods
as
such.
The
following
blog
post
comments
clearly
demonstrate
the
different
views:
The
origin
of
the
results
agenda
lies
in
a
mistrust
that
eats
like
a
cancer
into
aid
agencies'
capacity
to
make
a
difference.
I
am
not
convinced
the
emphasis
on
results
will
solve
the
problem
of
trust.
On
the
contrary,
it
risks
making
things
worse.
The
results
rhetoric
gets
exaggerated
by
bureaucratic
systems
and
by
those
middle-‐level
managers
with
little
country-‐level
experience
who
are
forcing
grantees
and
development
partners
into
straitjackets
that
constrain
them
from
helping
transform
the
lives
of
people
in
poverty.
(Eyben,
2011)
A
results
agenda,
as
long
as
the
right
results
are
being
pursued,
can
help
to
rebalance
inequalities
of
power
and
make
the
actions
and
decisions
of
the
powerful
more
transparent.
It
helps
people
to
know
what
the
objectives
of
decision-‐makers
are
–
and
so
to
argue
that
they
should
be
different,
if
that's
the
case;
and
also
to
hold
people
to
account
for
their
success
or
failure
to
meet
those
objectives.
Without
measurement,
there
can
be
no
accountability.
(Melameb,
2011)
Another
interesting
blog
questioning
the
results
agenda
is
the
“Aid
on
the
Edge
of
Chaos,
edited
by
Ben
Ramalligam”34.
On
this
blog,
discussions
include
those
on
result
focus,
and
the
complexity
of
development
cooperation.
One
of
the
recurring
themes
is
that
the
results
agenda
–
as
it
is
being
shaped
and
implemented
by
development
cooperation
agencies
-‐
is
actually
in
conflict
with
the
recommendations
provided
by
the
literature
regarding
the
need
for
more
flexibility
and
responsiveness
to
changing
contexts
in
developing
countries.
Also
Ben
Ramalligam
has
questioned
why
the
results
agenda
as
such
does
not
take
this
criticism
into
account,
and
has
promoted
that
“we
should
demand
the
same
results
of
the
’results
agenda’
as
it
demands
of
every
other
agenda”.
33
See
the
discussion
by
Duncan
Green,
Rosalind
Ayben
and
Claire
Melameb
at:
http://www.oxfamblogs.org/fp2p/?tag=results
34
http://aidontheedge.info/

44
7.6. Current political debate and The Busan Outcome Document:
As
stated
earlier,
the
results
agenda
has
received
a
renewed
focus,
and
was
a
top
priority,
at
least
among
traditional
donors
such
as
the
UK
and
Sweden,
during
the
HLF-‐4
meeting
in
Busan.
The
latest
political
rhetoric
from
the
UK
and
Sweden
has
been
quite
clear
on
this.
The
Swedish
Minister
for
Development
Cooperation,
Gunilla
Carlsson,
states
that:
“If
we
cannot
transparently
and
systematically
report
on
how
our
aid
budget
is
spent,
and
what
is
achieved
in
the
form
of
results,
the
credibility
of
development
cooperation
itself
will
be
undermined”35.
The
UK
State
Secretary
emphasised
in
his
speech
Results
for
Change
that
his
“number
one
priority
is
for
us
to
start
delivering
the
results
that
will
change
the
world”36.
In
the
speech
he
very
strongly
declared
that
he
believes
that
the
new
results
agenda
is
a
start
of
a
new
paradigm
across
the
development
community
–
a
paradigm
that
focuses
with
intensity
on
results,
that
places
evidence
above
ideology,
that
welcomes
external
scrutiny,
embraces
radical
transparency,
opens
its
doors
to
fresh
ideas
and
to
new
partnerships,
that
injects
the
dynamism
of
the
private
sector
into
its
DNA,
that
acts
as
a
critical
friend
to
its
partners,
and
that
directs
aid
on
the
basis
of
performance.
At
the
recent
High
Level
Meeting
in
Busan
the
results
agenda
received
a
renewed
push.
In
the
Busan
Outcome
Document,
the
Busan
Partnership
for
Effective
Development
Cooperation,
it
is
clearly
stated
that
sustainable
development
results
are
the
end
goal
for
development
co-‐operation
commitment,
but
that
development
cooperation
should
only
be
seen
as
part
of
the
solution;
it
plays
a
catalytic
and
indispensable
role
in
supporting
poverty
eradication,
social
protection,
economic
growth
and
sustainable
development.
The
participating
actors
agreed
to
strengthen
their
efforts
to
achieve
concrete
and
sustainable
results.
In
the
document
it
is
stated
that
this
involves
improved
management
for
results,
monitoring,
evaluating
and
communicating
progress,
as
well
as
scaling
up
our
support,
strengthening
national
capacities
and
leveraging
diverse
resources
and
initiatives
in
support
of
development
results.
Agreements
were
made
to
develop
joint
risk
management
frameworks
with
providers
of
development
cooperation,
and
to
develop
transparent,
country-‐led
and
country-‐level
results
frameworks
and
platforms
in
countries
where
the
partner
initiates
such
approaches.
The
frameworks
should
be
adopted
as
a
common
tool
among
all
concerned
actors
to
assess
performance
based
on
a
manageable
number
of
output
and
outcome
indicators
drawn
from
the
development
priorities
and
goals
of
the
developing
country.
Providers
of
development
cooperation
agreed
to
minimise
their
use
of
additional
frameworks,
refraining
from
requesting
the
introduction
of
performance
indicators
that
are
not
consistent
with
countries’
national
development
strategies.
It
was
also
agreed
that
the
actors
would
jointly
implement
a
global
action
plan
to
enhance
capacity
for
statistics
to
monitor
progress,
evaluate
impact,
ensure
sound
results-‐focused
public
sector
management,
and
highlight
strategic
issues
for
policy
decisions.
In
the
document
it
is
stated
that
the
type
of
global
monitoring
to
be
implemented
will
be
agreed
upon
by
June
2012,
indicating
that
the
previous
global
monitoring
done
through
the
Paris
Declaration
Monitoring
Survey
and
its
subsequent
indicators
still
are
to
be
defined.
35
http://www.naiforum.org/2011/09/taking-‐international-‐development-‐into-‐the-‐21st-‐century/
36
http://www.dfid.gov.uk/Media-‐Room/Speeches-‐and-‐articles/2011/Andrew-‐Mitchell-‐on-‐Results-‐for-‐Change/?utm_&&&

In
the
commitments,
emphasis
is
put
on
the
development
of
results
frameworks
in
partner
countries,
and
that
these
should
be
developed
in
countries
that
take
the
initiative.
Development
of
country-‐
owned
result
frameworks
requires
strong
ownership
of
the
process
from
the
partner;
and
these
frameworks
will
only
be
effective
if
there
also
is
a
willingness
from
donors
to
use
the
frameworks
as
the
basis
for
their
support.
In
fact,
these
types
of
frameworks
are
already
in
place
in
countries
with
modalities
such
as
budget
support,
a
modality
that
has
moved
development
dialogue
closer
to
political
decision
making
in
partner
countries,
but
with
reducing
political
support
in
donors
countries
due
to
conceived
political
risks
involved
with
the
modality.
At
the
moment,
when
more
efforts
are
being
put
into
finding
new
mechanisms
and
actors
through
which
development
cooperation
can
be
channelled,
harmonisation
around
the
partner
country’s
processes
and
result
frameworks
become
even
more
important.
However,
the
political
discourse
in
Busan
and
in
development
cooperation
at
moment
is
perhaps
the
clash
between
traditional
donors
and
traditional
ways
of
implementing
development
cooperation,
and
new
actors
and
new
ways
of
providing
support
to
development
such
as
South-‐South
cooperation.
In
Busan
this
potential
conflict
shaped
the
discussion
on
the
outcome
document.
In
the
end
all
actors,
including
new
emerging
donors
such
as
China
and
India,
endorsed
the
guiding
principles
for
aid
effectiveness
that
were
outlined
in
the
Busan
Partnership
for
Effective
Development
Cooperation.
However,
paragraph
2
of
the
documents
includes
a
clause
that
states
that
“the
principles,
commitments
and
actions
agreed
in
the
outcome
document
in
Busan
shall
be
the
refrence
for
South-‐South
partners
on
a
voluntary
basis”.
45
In
sum,
when
one
analyses
the
initiatives
that
have
arisen
during
the
last
decade
the
following
pattern
emerge:
1. Different
financing
forms
and
incentives
that
lead
to
best
development
results
are
being
explored.
2. There
are
many
initiatives
supporting
partner
countries’
and
their
management
practices.
There
are
also
multiple
supportive
tools
that
clarify
best
practices
and
offer
“how
to”-‐
guidance
to
develop
results-‐oriented
Monitoring
and
Evaluation
systems
etc.
at
in
partner
countries.
These
“how
to”-‐guides
seem
to
be
less
prominent
on
the
donor
side.
3. Several
global
quality
measurement
initiatives
have
arisen
the
latest
few
years,
many
of
them
comparing
donor
performance.
More
initiatives
continue
to
arise,
but
at
the
same
time,
the
discussion
prior
to
Busan
seems
to
indicate
a
tiredness
of
global
measurements.
4. 3
IE,
Regional
Centers
for
Learning
on
Evaluation
and
Results
(CLEAR),
PARIS21
and
the
Big
Push
Forward
are
examples
of
initiatives
that
have
followed
as
a
reaction
to
result
management
implementation
practices.
These
initiatives
demonstrate
that
a
lot
of
forces
are
pushing
to
improve
partner
country
results
management
and
statistical
systems.
The
driving
force
behind
the
initiatives
comes
from
donors,
who
have
a
strong
pressure
to
demonstrate
results
in
their
programmes.

46
8. Recommendations and lessons provided in the reports
All
of
the
literature
reviewed
in
Chapter
5
provides
recommendations
and
lessons.
In
addition,
there
are
several
source
and
“How
to-‐guides”
that
provide
recommendations,
and
best
practice
principles
on
how
to
go
forward
(IEG,
MfDR,
Mayne).
In
this
chapter,
we
have
tried
to
capture
and
categorise
the
different
recommendations
and
lessons
from
the
reports.
In
the
literature
“lessons”,
“best
practices”,
“recommended
practices”,
or
“recommendations”
are
provided.
Best
practices
or
lessons
are
most
frequently
provided
at
a
principal
level.
Many
of
the
lessons
and
recommendations
relate
directly
to
the
shortcomings
raised
in
the
reports.
Therefore,
it
seems
as
if
there
are
insufficient
experiences
with
aspects
and
reforms
that
have
succeeded
in
practice
in
an
organisation.
In
this
chapter
we
will
thus
highlight
some
of
the
general
recommendations
on
how
one
should
think
and
thereafter,
in
bullet
points,
demonstrate
some
of
the
recommendations
on
how
one
should
act
to
change
current
faulty
practices.
Most
of
the
recommendations
is
derived
from
literature
on
donors.
Strong leadership and “buy-in” throughout the organisation
The
most
frequent
lesson
in
the
different
reviews
is
the
need
for
support
for
RBM
from
top
leadership,
and
a
“buy-‐in”
of
the
system
at
all
levels
within
the
organisation.
Without
strong
advocacy
from
senior
managers,
results
based
management
systems
are
unlikely
to
be
institutionalised
either
broadly
or
effectively
within
an
agency
(Binnedjikt).
The
UNDP
evaluation,
for
example,
states
that
“The
first
and
overarching
recommendation
addresses
the
need
to
capitalize
on
what
has
been
achieved
to
date
and
establish
a
stronger
culture
of
results.
The
success
of
this
is
not
dependent
upon
tools
and
systems,
but
leadership
and
direction”.
Several
critics
have
pointed
out
that
RBM
is
not
primarily
a
matter
of
technical
skills
(DAC,
UN),
but
a
question
on
organisational
culture.
In
building
such
a
culture
all
of
the
reviews
emphasise
the
importance
of
leadership
requesting
results
information
–
“what
is
measured
gets
done.
At
the
same
time,
another
general
lesson
is
that
without
a
“buy-‐in”
by
project/programme
management
and
partners,
the
performance
data
is
unlikely
to
be
used
in
operational
decision
making.
Imposed
top-‐
down
systems
may
lack
relevance
to
actual
project/programme
management,
may
not
sufficiently
capture
the
diversity
of
results,
and
may
even
lead
to
programme
distortions
as
managers
attempt
to
achieve
what
is
measurable,
rather
than
what
is
relevant.
Hence,
some
operational
level
flexibility
is
needed
for
defining,
measuring,
reporting,
and
using
results
data
that
will
be
appropriate
to
the
specific
project/programme,
and
to
its
country
setting.
(Binnedjikt)
In
conclusion,
field
managers
need
both
leadership
and
autonomy
if
they
are
going
to
manage-‐for-‐results.
How
is
this
done
in
practice?
Some
of
the
recommended
actions
for
fostering
stronger
RBM-‐
leadership
and
a
“buy-‐in”
throughout
the
organisation
are:
• Build
knowledge,
competence
and
understanding
of
RBM
through
training
of
top
leadership.
Ensure
that
top
management
visibly
leads
and
demonstrates
the
value
of
RBM
by

consistently
communicating
and
asking
questions
based
on
results
information.
They
should
be
clear
about
the
value
and
need
for
results
information,
and
maintain
how
it
is
used
to
improve
management.
47
• Support
managers
that
are
experimenting
with
new
approaches,
and
respect
managerial
freedom.
• Provide
occasions
where
managers
can
demonstrate
their
successful
results-‐based
management
practices,
and
display
the
accomplishments
of
those
programmes.
• Foster
and
support
peer
RBM
champions,
and
organise
events
for
peer
pressure.
• Support
resources
for
RBM
for
training
and
acquisition
of
RBM
expertise,
and
enable
a
central
unit
to
provide
RBM
support
to
managers
and
staff.
Base the donor system on national priorities and ownership
All
reviews
and
evaluations
recommend
a
stronger
focus
on
the
partner
relationship,
and
the
need
to
base
a
donor
results
system
on
partners’
system/s
and
priorities.
The
MfDR
source
book
is
perhaps
most
explicit
on
this
principle.
A
key
recommendation
is
therefore
to
focus
the
dialogue
on
results
for
partner
countries
–
at
all
phases
of
implementation
–
from
strategic
planning
to
completion
and
beyond.
Both
the
UNDP-‐evaluation
and
the
multilateral
review
conclude
that
it
is
of
the
utmost
importance
to
work
with
RBM
within
the
context
of
partner
country
policy
and
fiscal
environments.
The
multilateral
review
concludes
that:
“Attempts
to
implement
RBM
within
Multilateral
Development
Institutes
alone,
or
within
developing
countries
but
without
a
supportive
policy
and
fiscal
environment,
are
unlikely
to
succeed.“
Furthermore,
the
issue
of
ownership
by
partner
countries
is
raised
as
a
prerequisite.
The
MfDR
sourcebook
states
that
“the
programs
in
which
the
greatest
problems
arose
in
the
results
aspects
tended
to
be
those
with
poor
ownership
–
the
donors
had
played
too
strong
a
role
in
defining
the
program
for
the
partner
countries.
Implementing
agencies
in
the
countries
felt
that
they
were
not
sufficiently
vested
in
the
program
and
were
not
responsible
for
shaping
the
outcomes.”
OECD/DAC
has,
in
several
of
its
documents,
stated
that
a
focus
on
results
and
relinquishment
of
ownership
to
partners
requires
a
shift
in
mind-‐set.
In
the
MfDR-‐sourcebook,
CiDA
provides
examples
of
staff,
who,
they
declare,
were
trained
under
the
assumption
of
bilateral
control
-‐
e.g.
that
Canadian
companies
or
organisations
would
be
hired
to
directly
implement
projects
and
initiatives,
with
far
more
emphasis
on
process
than
results.
They
state
as
an
experience
that
“the
old
mindset
of
bilateral
control
lives
on;
but
it
needs
to
be
replaced
–
in
practice
as
well
as
theory
–
with
principles
of
partnership,
accountability,
and
trust”.
Some
reviews
recommend
actions
in
order
to
increase
ownership
by
partners
and
a
shift
in
mindset.
These
recommendations
include:
• Actually
using
the
partner
country
systems.
• Training
to
augment
pre-‐existing
mind-‐sets.
• Taking
some
calculated
risks
that
imply
partner
trust.
• Using
participatory
processes
that
involve
partners,
beneficiaries,
and
stakeholders
in
all
phases
of
results-‐based
management
–
e.g.,
in
strategic
planning
exercises,
in
developing
performance
measurement
systems,
and
in
results-‐based
decision-‐making
processes.

48
Clarity and simplicity in the results reporting systems
Several
of
the
reports
highlight
the
importance
of
having
a
policy
framework
that
supports
result
reporting
systems,
and
clearness
regarding
the
extent
and
limitations
of
the
RBM-‐system.
Several
reviews
clearly
recommend
rules
and
regulations
for
planning,
budgeting,
monitoring,
and
evaluation
where
results
information
is
to
be
used.
Another
commonly
cited
lesson
is
an
emphasis
on
the
clarity
on
roles
and
responsibilities
in
regard
to
how
data
should
be
collected,
analysed
and
used.
Many
reports
mention
that
the
user-‐friendliness
of
the
system,
and
the
RBM-‐system
itself,
should
be
regularly
reviewed
and
updated.
An
important
aspect
of
use,
is
that
the
RBM
system
is
not
too
complicated.
The
following
actions
are
recommended
for
increasing
clarity
and
simplicity:
• Begin
with
simple
systems
and
use
the
information
for
dialogue
to
improve
the
process.
Proceed
gradually
and
with
modesty.
• Do
not
just
add
on,
but
clarify
which
old
procedures
and
requirements
can
be
dropped,
or
provide
adequate
additional
resources.
• Set
out
reasonable
expectations
for
RBM.
• Allow
sufficient
time
and
resources.
Finally,
in
the
MfdR
Sourcebook
Canadian
Cida
underscores
the
importance
that
funding
cannot
be
managed
merely
with
respect
to
the
achievement
of
targets.
Longer-‐term
goals
and
potential
impact
must
be
articulated,
and
dialogue
on
changes
and
the
process
is
important
for
program
implementation.
Ensure usage and foster a culture of results
All
reports
refer
to
the
lesson
of
ensuring
that
performance
information
is
not
used
just
for
reporting,
but
for
management
learning,
improved
analysis,
and
decision-‐making.
Some
of
the
reviews
(Binnedjikt,
Multi)
connect
this
lesson
with
the
tension
between
reporting
and
management
learning,
and
therefore
recommend
that
agencies
try
to
balance
their
different
needs.
Most
reports
address
the
importance
of
the
need
to
improve
the
use
of
data.
Some
note
that
results-‐
based
management
needs
to
be
clearly
correlated
with
results-‐based
budgeting.
The
Multi-‐review
states
that
“Resources
and
recognition
needs
to
flow
to
those
individuals,
units,
sectors
and
countries
with
the
best
record
of
managing
for,
and
delivering,
results.“
Some
recommendations
to
increase
usage
include:
• Begin
with
pilot
efforts
to
demonstrate
effective
RBM
practices.
• Give
managers
the
autonomy
to
manage
for
results,
while
also
holding
them
accountable.
• Have
rewards
for
groups
as
well
as
individuals.
• Provide
a
variety
of
support
mechanisms.
• Encourage
learning
through
experience.
• Anticipate
and
avoid
misuses
of
performance
measurement
systems.
(E.g.
dishonest
reporting,
concentration
on
those
activities
that
are
most
easily
measured,
holding
managers
accountable
for
results
beyond
their
control.)
In
sum,
many
of
lessons
and
recommendations
point
to
the
fact
that
RBM
cannot
succeed
without
leadership,
resources,
incentives,
clear
and
simple
support
systems
and
a
shared
understanding
of
the
purpose
of
RBM
–
i.e.
that
results
information
should
be
used
for
improved
decision
making
and
learning.

49
9. Conclusions
The
desire
to
achieve
results
in
development
cooperation
has
always
been
part
of
the
aid
agenda,
and
various
methods
and
approaches
have
been
implemented
to
achieve
this
end.
During
the
1970s
and
80s,
results
management
was
applied
to
project
aid
through
methods
such
as
the
Logical
Framework
Approach.
Experience
and
evidence
preceding
the
Paris
Declaration
(2005)
indicates
that
a
fragmented
aid
landscape,
with
donors
working
on
their
own
projects,
did
not
lead
to
sustained
development
results;
there
was
an
emerging
need
for
donors
to
change
their
methods
of
thinking
and
working.
Joint
global
goals
on
WHAT
to
do
and
HOW
to
achieve
the
ultimate
objective
of
development
cooperation,
i.e.
poverty
reduction,
were
agreed
upon
via
the
Millennium
Development
Goals
and
the
Paris
Declaration
on
Aid
Effectiveness.
Results
management
became
an
integral
part
of
this
development.
During
the
1990s,
it
transformed
from
being
project-‐specific
to
a
more
organisation-‐
and
programme-‐wide
level.
During
the
2000s,
a
results
focus
has
increasingly
moved
towards
including
organisations
and
systems
in
partner
countries.
The
basic
idea
behind
the
results
agenda
–
i.e.
that
we
need
performance
information
to
understand
and
adapt
to
development
contexts
and
to
make
informed
choices
–
seems
uncontested.
However,
even
the
shallow
review
of
studies
cited
in
this
paper
refers
to
severe
difficulties
with
agenda
implementation
across
different
organisations,
and
in
different
contexts
within
development
cooperation.
The
reviews
and
evaluations
we
have
analysed
generally
refer
to
these
as
implementation
problems,
and
provide
a
number
of
recommendations
for
how
to
overcome
these
challenges.
It
is
evident
that
differences
in
approaches
are
grounded
in
competing
world
views
(or
contrasting
epistemological
and
ontological
assumptions).
The
theoretical
perspective
shapes
the
way
change
is
understood
(linear
or
complex),
how
projects
and
programmes
should
be
designed
(based
on
“best
practices”
of
“what
works”
or
on
a
contextual
analysis
involving
the
change
agents),
and
thereby
methods
for
evaluating
and
managing
for
results.
However,
we
believe
it
is
important
to
also
look
underneath
the
surface,
at
the
theoretical
perspective
behind
the
tools
and
methods.
In
this
review
we
have
studied
reviews
and
evaluations
that
analyse
how
mainstream
results-‐based
management
theory
has
been
implemented.
We
have
found
indications
that
enhanced
results
focus
and
results
management
have
led
to
improved
policy
coherence
and
planning,
particularly
in
larger
partner-‐led
programmes.
Some
studies
also
indicate
that
a
results
focus
at
the
programme
and
project
level
has
improved
national
monitoring
systems,
and
that
the
strength
of
the
monitoring
system
is
positively
correlated
with
higher
project
outcome
ratings.
At
the
donor
level,
the
OECD/DAC
suggests
that
an
increased
results
focus
has
helped
to
broaden
the
support
for
aid,
and
has
helped
to
drive
a
cultural
change
regarding
how
programmes
are
managed.
At
the
same
time,
the
literature
points
to
a
wide
variety
of
results
management
models,
along
with
corresponding
difficulties
with
their
application
and
use.
The
difficulties
can
be
found
at
all
levels,
but
perhaps
the
most
severe
challenges
are
found
at
the
donor
institutional
level.
These
include
troubles
with:
(i)
aggregating
data,
(ii)
measuring
outcomes
and
impact
in
the
short
term,
(iii)
selecting
appropriate
indicators
to
measure
relevant
outcomes,
and
(iv)
attributing
results
to
aid-‐funded
projects
and
programmes.
Organisations
report
that
results
management
has
often
been
introduced

50
as
an
add-‐on
to
other
management
reforms
and
policies,
implying
that
systems
overextend
and
complicate
their
administrative
and
human
resources.
Also,
it
is
reported
that
performance
information
is
not
used
for
the
purposes
intended
by
results
management
-‐
improved
analysis,
planning
and
decision-‐making
-‐
but
is
instead
mechanically
used
for
accountability
and
control
purposes.
On
the
whole,
implementing
the
results
agenda
in
development
cooperation
has
proved
difficult.
This
seems
to
be
because:
1.
Results-‐based
management
is
complex
to
handle
at
organisational
and
human
level
It
is
repeatedly
reported
that
the
results
management
perspective,
in
its
more
holistic
form
results-‐based
management,
is
ambitious.
One
is
asking
mostly
non-‐results-‐oriented
organisations
to
institute
large
measures
of
change
in
planning,
reporting,
and
implementation.
When
introduced
at
an
organisational
level,
there
seems
to
be
a
tendency
to
interpret
the
results
perspective
as
mechanistic
and
deterministic,
in
contrast
with
analytic
and
responsive-‐to-‐change
notions.
The
review
suggests
that
organisations
often
fail
to
make
out
workable
results
oriented
models,
or
even
put
performance
information
to
use
as
intended
by
results
management.
2.
Different
purposes
create
conflicts
in
application
Uncertainty
in
the
development
community
about
what
results
management
means
and
is
used
for
is
likely
to
stem
from
a
variety
of
motivations
and
objectives.
A
diversity
of
objectives
includes
disbursement
levels,
accountability
(domestic
as
well
as
mutual),
control,
and
planning
and
learning.
For
aid
agencies,
there
is
a
conflict
in
the
dual
objective
of
supporting
partner
country
result
management
systems
while
simultaneously
improving
their
own.
The
representation
of
“results-‐oriented
practices”,
in
this
context,
has
become
ambiguous.
3.
Results
management
goes
against
management
practices
centred
on
control
and
process
It
appears
that
when
the
results
perspective
confronts
management
practices
centred
on
control
and/or
process,
managers
are
inclined
to
look
away
from
the
outcome
level,
and
turn
their
attention
to
what
is
easily
measurable
(normally
indicators
found
on
the
levels
of
input-‐
activity-‐output).
Specifically,
it
seems
that
the
call
for
a
results
focus
on
“outcomes”
has
contributed
to
risk-‐averse
behaviour
by
donor
agencies.
The
challenge
of
introducing
“new”
management
perspectives
also
seem
to
be
aggravated
in
partner
countries,
where
the
agenda
is
at
times
seen
as
driven
by
donor
country
governments,
and
with
low
demand
in
partner
countries.
4.
The
systemic
and
cultural
demands
put
on
development
partners
Working
with
monitoring
and
evaluation
in
environments
where
information
is
scarce,
and
national
demand
for
performance
information
to
influence
policy
and
decision-‐making
is
weak,
presents
natural
challenges
for
results
orientation
and
mutual
accountability.
It
is
also
evident
that
donors
face
difficulties
in
simultaneously
strengthening
and
aligning
with
development
partner
systems
for
planning
and
performance
assessment.

On
these
grounds,
among
others,
the
results
agenda
has
been
questioned
to
the
point
of
suggesting
that
the
entire
idea
is
flawed.
It
may
be
that
basic
notions
lead
to
misuse
on
a
level
surpassing
any
potential
benefits,
and
that
it
is
beyond
our
reach
to
increase
aid
effectiveness.
However,
that
reasoning
requires
an
alternative.
From
our
somewhat
limited
vantage
point,
suggested
remedies
tend
to
rely
on
the
notions
of
improved
analysis
and
learning
from
past
experiences.
Assuming
that
the
basic
idea
of
an
analytic,
performance-‐oriented
perspective
on
management
is
valid,
the
question
that
follows
is:
how
to
activate
it?
And
with
regard
to
development
cooperation:
is
it
possible
to
implement
the
agenda
successfully
this
time?
Today
the
agenda
is
being
pushed
stronger
than
ever.
There
is
an
increased
global
and
domestic
demand
to
demonstrate
the
results
of
development
cooperation.
The
effects
of
traditional
forms
of
development
cooperation
are
being
questioned
publicly.
New
modalities
and
actors
are
entering
the
scene
and
thus,
the
complexity,
number
of
actors,
and
ways
of
doing
business,
has
never
been
greater.
Results
requirements
are
increasingly
being
used
as
a
condition
for
funding:
“without
results
-‐
no
funding
and
no
collaboration”.
51
Obviously,
and
as
this
review
indicates,
the
demonstration
of
results
and
effects
of
development
cooperation
is
complex
and
difficult.
Excessive
faith
that
results-‐based
management
and
its
control
aspects
can
solve
this
complexity
seems
to
be
counterproductive.
The
following
image
could
demonstrate
this
reasoning:
In
this
image,
from
left
to
right,
marginal
utility
from
results
management
increases
where
policy
and
programmes
are
based
on
clear
objectives,
thus
confirming
that
a
certain
focus
on
results
is
necessary.
Experience
infers
that
at
a
certain
point,
however,
complexity
and
differing
interpretations
blur
the
picture.
At
this
point,
various
choices
are
available.

Applying
results
management
for
control
and
command
purposes
seems
to
lead
towards
a
mechanistic
interpretation
and
use
of
results
management
tools.
This
appears
to
be
the
current
situation
as
reported
upon
by
the
various
evaluations
and
reviews
of
donor
agencies
–
i.e.
the
control
agenda
has
taken
over
and
led
to
what
for
example
Natsios
calls
“obsessive
measurement
disorder”.
The
alternative
might
be
to
use
results
management
for
learning
or
analytic
purposes.
In
this
scenario,
performance
information
is
used
to
achieve
a
better
foundation
for
policy
and
decision-‐
making
(which
has
seldom
been
the
case
today,
according
to
various
reviews).
This
approach
could
either
be
based
on
the
same
theoretical
assumptions
that
underpin
the
current
mainstream
paradigm
of
results-‐based
management,
or
be
grounded
on
another
theoretical
foundation,
as
offered
by
late
modern
schools
such
as
complexity
theory.
In
the
first
case,
rigorous
impact
evaluations
(as
offered
by
3ie)
and
improved
systems
for
systematic
organisational
learning
would
be
suitable
tools.
In
the
second
case,
more
process-‐oriented
approaches
to
learning
and
decision
making
would
need
to
be
adopted,
such
as
participatory
utilisation-‐focused
evaluation
or
outcome
mapping.
52
Assuming
that
RBM
is
not
a
model
but
rather
a
perspective
on
management,
the
introduction
of
more
nuance
in
its
application
–
susceptible
to
various
objectives,
alternative
ways
of
analysis
and
lessons
learned
from
previous
application
–
perhaps
presents
ways
forward.
Our
final
word
is
therefore
to
keep
sight
of
the
need
for
sound
analyses
to
guide
development
cooperation,
and
to
explore
how
results-‐oriented
perspectives
can
be
developed
to
find
a
workable
approach.
Results
can
be
achieved
by
creating
trust,
learning
from
each
other,
dialogue,
and
human
relations.
Such
values
are
probably
better
promoted
if
the
results
agenda
can
be
made
to
work
for
them.

53
Annex 1. Some suggestions for further research
1. Country
level
studies
and
studies
on
cost-‐effectiveness
of
the
results
agenda.
Cost
effectiveness
and
value
for
money
are
usually
presented
as
key
principles
for
results
management.
Results
information
is
collected
and
disseminated
in
order
to
increase
cost
effectiveness
and
value
for
money.
However,
our
review
indicates
that
less
emphasis
has
been
put
on
discussions
about
the
cost-‐implications
of
the
results
agenda
as
such.
Data
collection
and
verification
systems
are
normally
costly.
Several
of
the
recommendations
from
the
reports
recommend
increased
staff
and
increased
capacity
for
data
collection,
support
mechanisms,
analysis
capacity,
etc.
Also,
it
seems
that
resources
for
control
and
evaluation
have
increased
in
many
donor
agencies.
However,
several
of
the
efforts
seem
to
be
put
on
the
donor
side.
In
partner
countries,
fragmentation
and
separate
results
management
systems
still
seem
to
be
common.
There
are
a
few
examples
(besides
budget
support
and
programme-‐based
approaches)
where
donors
have
jointly
come
together
to
support
result
measurement
systems
in
partner
countries,
and
where
they
at
the
same
time
have
reduced
their
own
administrations.
Therefore,
we
suggest
more
research
of
how
results-‐based
management
has
been
applied
by
partner
countries,
including
cost-‐effectiveness
aspects
of
applying
the
agenda.
2. How
the
results-‐based
management
model
has
been
understood
and
applied
in
different
organisations
and
contexts.
Different
organisations
have
understood
and
applied
the
model
differently.
Few
comparative
research
and
studies
are
available
on
the
actual
understanding
and
application
of
the
model.
Questions
arise
regarding
if
certain
“models”
could
be
viewed
as
more
effective
than
others,
and
whether
results
management
actually
leads
to
increasing
aid
effectiveness
or
increased
achievement
of
results.
3. The
results
agenda
in
the
new
aid
architecture
–Traditional
aid
versus
aid
provided
by
new
donors’
and
actors’.
Conditionalities
on
results
are
mainly
being
pushed
by
traditional/Western
donors,
while
traditional
aid
is
being
publicly
questioned.
If
results
(as
formulated
by
the
donors)
are
not
achieved
–could
this
be
the
blind
alley
for
traditional
aid?
How
do
the
new
donors
view
results
and
act
for
increasing
results
of
their
development
cooperation?
What
kind
of
effects
can
be
demonstrated
from
different
results-‐based
financing
initiatives?
Are
they
more
effective
in
achieving
results
than
traditional
aid?
4. How
do
aid
managers
deal
with
the
multiple
accountabilities
that
form
part
of
the
results
agenda,
and
how
does
the
results
agenda
affect
the
behaviour
of
organisations
funded
by
development
cooperation.
If
all
evaluations
show
that
results
information
is
not
used
for
the
purpose
of
decision-‐making,
what
is
it
that
aid
managers
are
basing
their
decisions
on?
And,
how
do
aid
managers
interpret
what
they
are
doing,
how
their
management
practices
form
part
of
the
results
chain
and
what
they
seek
to
achieve?

54
Literature
Binnedjikt,
Annette
(2000)
Results
Based
Management
in
the
Development
Co-‐operation
Agencies
–A
Review
of
Experience,
OECD/DAC
Working
Party
on
Aid
Evaluation
Clemens,
Michael.
Moss,
Todd
(2007)
The
ghost
of
0,7
per
cent
origins
and
relevance
of
the
international
aid
target,
International
Journal
of
Development
Issues
Vol
6.
No
1
Curtis,
Donald
(2004)
‘How
we
think
they
think’:
Thought
styles
in
the
management
of
international
aid,
Public
Administration
and
Development
24,
415-‐423
Delay,
Simon
(2005)
Why
does
aid
management
in
Samoa
succeed?
A
note,
Public
Administration
and
Development
25,
433-‐435
Drucker,
Peter
(1964);
Managing
for
results
Drucker,
Peter
(1954),
The
Practice
of
Management
Earl,
Sarah;
Carden,
Fred;
Smutylo,
Terry
(2001)
Outcome
Mapping
-‐
Building
Learning
and
Reflection
Into
Development
Programs,
International
Development
Research
Centre
Eichler,
Rena
(2010)
RBF
-‐
Something
New?
The
World
Bank
Group.
http://www.rbfhealth.org/rbfhealth/blog/2010/06/08/346/rbf-‐something-‐new.
Easterly,
William
()
Rhetoric
versus
Reality:
The
Best
and
Worst
of
Aid
Agency
Practices
Development
Research
Institute,
New
York
University
Ergo,
Puhl
(2011)
TrAid+
Channeling
Development
Assistance
to
Results,
Working
Paper
247,
Centre
for
Global
Development
European
Commission/DG
Development
(2002)
Guidelines
for
the
use
of
indicators
in
country
performance
assessment
Eyben,
Rosalind
(2011)
Stuff
happens’:
The
risks
of
a
results
agenda,
From
Poverty
to
Power
–Oxfam
Blog
edited
by
Duncan
Green.
http://www.oxfamblogs.org/fp2p/?p=4743
Eyben,
Rosalind
(2010)
Hiding
Relations:
The
Irony
of
‘Effective’
Aid,
European
Journal
of
Development
Research
Vol
22,
3,
382-‐387
Flint,
Michael
(2003)
Easier
said
than
done
–A
review
of
Result
Based
Management
in
multilateral
development
institutions
Forrester
and
McLoughlin
(1998)
Sector
Wide
Approaches
and
Financial
Accountability
Gasper,
Des
(2000)
Evaluating
the
‘Logical
Framework
Approach’
Towards
learning-‐oriented
development
evaluation,
Public
Administration
and
Development
20,
17-‐28
Glennie,
Jonathan
(2011)
Is
Cash
on
Delivery
future
for
Foreign
Aid?
The
Guardian
Poverty
Matters
Blog.
http://www.guardian.co.uk/global-‐development/poverty-‐matters/2011/may/13/cash-‐delivery-‐
future-‐results-‐based-‐aid
Goss
Gilroy
Management
Consultants
(2011)
Results
Reporting
in
donor
agencies

55
GDPRD,
FAO
and
WB
(2008)
Tracking
results
in
agriculture
and
rural
development
–A
sourcebook
Guijt,
Irene
(2008)
Critical
readings
on
assessing
and
learning
for
social
change,
IDS
Hatton,
Michael.
Schroeder,
Kent
(2007)
Result
Based
Management:
friend
or
foe?,
Development
in
Practice,
Volume
17,
Number
3
Hervio,
Perez
and
Emblad
(2000)
Conditionality
Reform:
The
Burkina
Faso
Pilot
Case,
SPA
Task
Team
on
Contractual
Relationships
and
Selectivity
Ireland,
Mark.
Mc
Gregor,
Allister.
Saltmare,
Douglas.
(2003)
Challenges
for
donor
agency
country-‐
level
performance
assessment:
A
review,
Public
Administration
and
Development
23,
419-‐431
IDA
(2006)
A
review
of
the
use
of
Out
–put
based
Approaches,
International
Development
Association
Sustainable
Development
Network
IEG
(2008),
Annual
Review
of
Aid
Effectiveness
Jones,
Harry
(2011)
Taking
responsibility
for
complexity-‐
How
implementation
can
achieve
results
in
the
face
of
complex
problems,
Working
Paper
330.
ODI
IDD
and
Associates
(2006)
Evaluation
of
General
Budget
Support:
Synthesis
Report
INTRAC,
O.
Bakewell,
A.
Garbutt,(2005)
The
use
and
abuse
of
the
Logical
Framework
Approach
–
A
Review
of
International
Development
NGO:s
Experiences
Knack,
Stephen.
Halsey,
Rogers.
Eubanck,
Nicholas.
(2010)
Aid
quality
and
donor
rankings,
World
Bank
Policy
Research
Working
Paper
No.
5290
Lindgren,
Lena
(2006)
Utvärderingsmonstret:
kvalitets-‐
och
resultatmätning
i
den
offentliga
sektorn,
Studentlitteratur
Mackay,
Keith
(2007)
How
to
build
M&E
systems
to
support
better
Government
–International
Evaluation
Group
(IEG),
World
Bank
Mathiason,
John
R
(2008)
Who
controls
the
machine,
III:
Accountability
in
the
Results
Based
Revolution
Public
Administration
and
Development
24,
61-‐73
Mayne,
John
(2007)
Best
Practices
in
Results-‐Based
Management:
A
Review
of
Experience.
A
Report
for
the
United
Nations
Secretariat
Mayne,
John
(2007);
Challenges
and
lessons
in
implementing
Results-‐based
Management,
Evaluation
Vol.
13,
2007
Melameb,
Claire
(2011)
If
not
results,
then
what?
The
risks
of
not
having
a
results
agenda.
From
Poverty
to
Power
–Oxfam
Blog
edited
by
Duncan
Green.
http://www.oxfamblogs.org/fp2p/?p=4756
Mitchell,
Andrew
(2011)
Results
for
change
–speech.
http://www.dfid.gov.uk/Media-‐
Room/Speeches-‐and-‐articles/2011/Andrew-‐Mitchell-‐on-‐Results-‐for-‐Change/?utm_&&&
Ministry
for
Foreign
Affairs
in
Finland
(2011)
Evaluation
of
Results
Based
Approach
in
Finnish
Development
Cooperation,
2011:2

56
Natsios,
Andrew
(2010)
The
clash
of
and
Development,
Centre
for
Global
Development,
www.cgdev.org/content/publications/detail/1424271
Odén,
Bertil
(2006)
Biståndets
idéhistoria
–Från
Marshallhjälp
till
milleniemål.
Studentlitteratur.
OECD
(1997)
In
Search
of
Results,
Public
Management
Practices,
1997
OECD/DAC
(2008)
Accra
Agenda
for
Action
OECD/DAC
(2011)
Aid
Effectiveness
2005-‐2010,
Progress
in
Implementing
the
Paris
Declaration
OECD/DAC
(2004)
Memorandum
on
the
High
Level
Roundtable
on
Development
Results
in
Marrakech
OECD/DAC
(2002)
Memorandum
on
the
High
Level
Roundtable
on
Development
Results
in
Washington
OECD/DAC
(2010)
Incentives
for
aid
effectiveness
in
donor
agencies
–Good
Practice
and
self-‐
assessment
tool
OECD/DAC;
Peer
Reviews,
selected
editions
1998-‐2010
OECD/DAC
(2005,
2007,
2008);
Sourcebook
on
Managing
for
Development
Results
OECD/DAC
/2005);
Managing
for
Development
Results,
principles
in
action,
Working
Party
on
Aid
Effectiveness
OECD/DAC
(2006)
Emerging
good
practice
in
managing
for
development
results
OECD/DAC
(2009)
Managing
Aid
–Practices
of
DAC
Member
Countries
OECD/DAC
(2011)
Has
donors
behavior
changed
since
the
Paris
Declaration?
Insights
from
DAC
Peer
Reviews,
DCD/DAC
2011
(29)
Olsen,
I
T
(2009)
Result-‐based
Financing
in
the
Health
Sector
Experiences
from
Norway
and
from
low-‐income
Countries.
Norad,
Oslo
Patton
Michael
Quinn
(2008)
Utilization
focused
Evaluation,
Sage
Publications
PARIS21
(2009a),
PARIS21
at
Ten
–
Improvements
in
Statistical
Capacity
since
1999,
PARIS21,
Paris
Pearson
Mark
(2011)
Results
based
aid
and
results
based
financing:
What
are
they?
Have
they
delivered
results?
HLSP
Institute
Ramalingam,
Ben
(2011)
Aid
on
the
edge
of
chaos.
Blog
Post.
http://aidontheedge.info
Ramalingam,
Ben
and
Jones,
Harry
(2008)
Exploring
the
science
of
complexity
–Ideas
and
implications
for
development
and
humanitarian
efforts,
Working
Paper
285.
ODI
Robinson,
David
Finlay
(2010)
Peter
Drucker
and
the
challenge
of
tyranny,
Journal
of
Management
Development
Vol
29.
No
1
pp.
111-‐122
Rogerson,
Andrew
(2011)
What
if
development
aid
really
rewarded
results?
Revisiting
the
cash
on
delivery
(COD)
Aid
Model,
OECD/DAC
Development
Brief
1/2011

57
Rosenberg,
Tina
(2011)
A
pay-‐for
Performance
evolution.
The
New
York
Times.
http://opinionator.blogs.nytimes.com/2011/03/17/a-‐pay-‐for-‐performance-‐evolution/
Savedoff,
William
D
(2011)
Incentive
Proliferation?
Making
Sense
of
a
New
Wave
of
Development
Programs,
Centre
for
Global
Development
Sida
SEKA
(2006)
Slutrapport
för
SEKAs
Resultatredovisningsprojekt
Sida
(2004)
Holmgren,
Svensson;
Performance
management
at
Sida,
Internal
audit
05/05
Statskontoret
(2011)
Styrning
av
svensk
biståndspolitik,
En
utvärdering
Sundström,
Göran
(2003)
Stat
på
villovägar:
resultatstyrningens
framväxt
i
ett
historisk-‐institutionellt
perspektiv,
Stockholm
University
UNDP
(2007)
Evaluation
of
Result
Based
Management
at
UNDP,
Evaluation
Office
UN
(2008)
Results
Based
Management
at
the
United
Nations,
Report
of
the
Office
of
Internal
Oversight
Services
Werner
Meier
(2003)
Results-‐Based
Management,
Towards
a
Common
Understanding
Among
Development
Cooperation
Agencies,
Discussion
paper
for
CIDA
White,
Howard
and
Dijkstra,
Geske
(2003)
Programme
aid
and
development:
Beyond
conditionality.
Routledge
studies
in
development
economics.
Wood,
Bernard
(2011)
(Team
Leader)
The
Evaluation
of
the
Paris
Declaration,
Final
Report,
Phase
2,
www.oecd.org/dac/evaluationnetwork/pde
World
Bank
(2011)
A
new
instrument
to
advance
development
effectiveness:
Program
for
Results
Lending,
Investment
Lending
Reform
Operations
Policy
and
Country
Services
World
Bank
(2007)
Result
Based
National
Development
strategies:
Assessment
and
Challenges
ahead

Results based management in development cooperation

More Related Content

What's hot (7)

Viewers also liked (17)

Similar to Results based management in development cooperation (20)

Results based management in development cooperation