Monitoring Evaluation Accountability and Learning Powerpoint by Abraham

18 August 2025 1
MEAL Training
Monitoring, evaluation, accountability, and learning.

18 August 2025 2
 Introduction: MEAL in Projects – What is MEAL?
 1. Designing Logic Models
 2. Planning MEAL Activities
 3. Collecting MEAL Data
 4. Analyzing MEAL Data
 5. Using MEAL Data
Table of Contents

18 August 2025 3
Introduction: MEAL in Projects

18 August 2025 4
 Define the components, structure, and purpose of MEAL
 Explain the benefits of a strong MEAL system
 Describe the relationship between MEAL and project management
 Identify the five phases of MEAL
 Describe the ethical standards and principles relevant to MEAL
 Understand the importance of participation and critical thinking in MEAL processes.
Introduction Objectives

18 August 2025 5
What is MEAL?
Learning
Accountability
Evaluation
Monitoring

18 August 2025 6
 Monitoring: The continual and systematic collection of data to provide information about
project progress.
 Evaluation: The user-focused, systematic assessment of the design, implementation,
and results of an ongoing or completed project.
 Monitoring and evaluation differ from each other especially in the questions they ask.
Monitoring
Evaluation M&E: What do they
mean and how do
they differ?

18 August 2025 7
• Monitoring questions ask
“How many people or communities were reached”
 “Did project processes complete activities on time and on budget?”
 “Was the project successful in achieving the targets set for its intended outcomes?”
• Evaluation questions ask
 “Is the project reaching (or did it reach) those with greatest need? If not, why?”
 “Did the project effectively and appropriately invest its time and budget to conduct its activities?”
 “How have outcomes achieved varied by different groups within the target area?”
M&E Questions

18 August 2025 8
Monitoring and evaluation also differ in terms of purpose,
frequency, timing, and use of data.
Purpose: Tracking inputs,
activities and progress toward
achievement of agreed
outcomes and impacts
Frequency: Regular and
ongoing during project
implementation
Responsibility: Activities are
conducted by members of the
project team
Use of data: Informs timely
decision-making and short-term
corrective action in support of
adaptive management
Purpose: A systematic and
objective assessment of the
merit, value or worth of an
ongoing or completed project
Frequency: Periodic, one-off
events during and, if funding
permits, after project
implementation
Responsibility: Activities are
often externally led, although
they should involve the active
participation of project staff
Use of data: Identifies potential
course corrections and
contributes to longer-term
organizational learning.
M
o
n
i
t
o
r
i
n
g
E
v
a
l
u
a
t
i
o
n

18 August 2025 9
 Accountability: A commitment to balance and respond to the needs of all stakeholders
(including project participants, donors, partners and the organization itself) in project activities.
 Projects should embrace accountability by promoting 1) Transparent communications, 2)
Alignment with standards, 3) Responsiveness, and 4) Participation.
 Learning: Having a culture and processes in place that enable international reflection. The
aim of learning is to make smarter decisions.
 Projects should learn through 1) Incentivizing learning, 2) Encouraging a spirit of curiosity, 3)
Embedding learning processes, 4) Promoting adaptive management, and 5) Sharing
information.
Accountability
Learning
Accountability &
Learning: What are they
and how do they differ?

18 August 2025 10
MEAL in Practice
Phase 1:
Designing
logic models
Phase 2:
Planning
MEAL
activities
Phase 3:
Collecting
MEAL data
Phase 4:
Analyzing
MEAL data
Phase 5: Using
MEAL data both
externally and
embed learning.
• MEAL is present and ongoing during
every stage of a project.
• Project MEAL activities can be
organized into five phases, described
in the Figure.

18 August 2025 11
Five phases of the MEAL cycle
Includes theory of
change, results
framework, and
logical framework.
They are to
explain the
change the project
hopes to achieve,
how this will occur,
and how change
will be measured.
1. Designing
Logic Models
Integrate MEAL
activities,
budgets, and
calendars
together into a
larger project
plan.
2. Planning
MEAL Activities
Develop and use
tools to collect
high-quality data
that measure
progress, help
you make
decisions, and
learn.
3. Collecting
MEAL Data
Conduct data
analysis during
and after project
implementation
according to the
analysis plans
established
during the MEAL
planning phase.
4. Analyzing
MEAL Data
Data should be
used to inform
management
decisions,
communication
s, and promote
accountability.
5. Using MEAL
data

18 August 2025 12
Poorly implemented MEAL can cause potential problems. To mitigate these problems,
organizations utilize ethical principles to assure high levels of professional conduct, such as:
• Representation: all populations have the right to be counted and represented in the data.
• Informed consent: participants in activities must be voluntary. They have the right to be
informed about the process, how data will be used, and to receive the results of the activity.
• Privacy and confidentiality: data collection and storage practices should keep the
participant and their opinions private and confidential.
• Participant safety: participants should not face any security risk due to participating in data
collection efforts.
• Data minimization: MEAL data collected should be relevant and focused to the project
needs.
• Responsible data usage: projects should establish and follow policies to protect the data
they collect, establish procedures to ensure data is used appropriately, stored securely and
destroyed when no longer needed.
Ethical Standards in MEAL

18 August 2025 13
MEAL processes should incorporate a variety of external stakeholder perspectives. A
stakeholder is someone who, because of their position or role, has an interest in and/or
influence on the project.
MEAL themes: Participation
Why engage
stakeholders in
MEAL planning
and
implementation
?
Ensure that
MEAL findings
are relevant to
the local
context.
Increase
local-level
capacity in
MEAL.
Contribute to
improved
communication and
collaboration
between project
actors working at
different levels of
project
implementation.
Promote a
more
efficient
allocation of
resources.
Increase
stakeholders’
understanding and
ownership of their
own program
strategy and
processes.

18 August 2025 14
MEAL processes require a consistent commitment to critical thinking. Critical thinking is a
process of thinking that is clear, rational, open to different opinions, and informed by evidence.
It also helps reduce bias.
To be successful, critical thinking requires that project teams apply the following behaviors:
 A willingness to identify the assumptions that shape your thinking and influence your actions.
 A desire to test the extent to which your assumptions are correct and well-founded.
 A capacity to ask thoughtful questions to pursue deeper understanding.
 An openness to multiple, sometimes conflicting, perspectives reflecting different expertise, experiences
and evidence.
 A commitment to reflection and analysis to inform actions.
MEAL themes: Critical Thinking

18 August 2025 15
Kahoot!
https://kahoot.it/
(Chapter 1)

18 August 2025 16
1. Designing Logic Models
Phase 1:
Designing
logic models
Phase 2:
Planning
MEAL
activities
Phase 3:
Collecting
MEAL data
Phase 4:
Analyzing
MEAL data
Phase 5: Using
MEAL data both
externally and
embed learning.

18 August 2025 17
 Describe how project logic models contribute to establishing a strong foundation for MEAL
 Compare and contrast the components, structure and purpose of theories of change, results
frameworks and Logframes
 Explain the purpose of identifying assumptions in project logic models
 Interpret the vertical and horizontal logic of Logframes
 Understand the characteristics of a SMART indicator
 Identify the most common measurement methods and when they are used
Objectives

18 August 2025 18
Logic Model
How do you
believe
change will
take place?
What is the
desired impact?
What
assumptions
need to hold
true for the
change to
occur?
How will you
measure and
track progress?
A logic model is a snapshot of how your
project is supposed to work. In other words, it
is a systematic, visual way to present a
summarized understanding of a project and
how it works.
• It articulates the desired long-term change
and maps what needs to happen for the
change to be achieved.
• A logic model may be a ToC, RF, logframe.
• It provides an overview that addresses
questions like:

18 August 2025 19
Progression of Logic Models
Theory of Change
Maps out the:
• Intended long-term change
• Interconnecting relationships
• Underlying assumptions and
supporting evidence
• Contributions from non-project
stakeholders that are needed for
change to occur.
Results Framework
Builds on the theory of change
by mapping out the:
• Project hierarchy of
objectives, including
objectives statements for
different levels of the project
• Causal logic of the project
results into a series of it-then
relationships.
Logical Framework
Builds on the theory of change
and the results framework by
mapping out the:
• High-level MEAL framework,
including indicators and means
of verification
• Assumptions that need to be in
place for the causal logic to
hold true.

18 August 2025 20
 It is a comprehensive and visual
description of how and why a desired
change is expected to happen.
 It should be wide-reaching, a
comprehensive analysis of needs,
assets, opportunities, and the
operating environment, and should
draw on stakeholder perspectives
and local knowledges.
 When developing a ToC, the process
should be participatory and evidence-
based.
Theory of Change
Visualize complex data and ideas in an
image that is easier to understand.
Identify the full range of changes
needed to achieve the intended impact.
These include changes that are
implemented by other stakeholders.
Recognize non-linear change.
Make explicit the assumptions, i.e. the
potential risks that could disrupt the
logic of the project.
Prompt discussion and participation by
opening up space to ask questions,
challenge assumptions and suggest
alternatives.
T
h
e
o
r
y
o
f
C
h
a
n
g
e

18 August 2025 21
ToC utilizes the following logic:
 Preconditions are the requirements that must exist for the long-term change to take place. Domains
of change are the broad strategic areas of intervention. Pathways of change identify the connections
between preconditions, how they relate to each other and in what order. Multiple pathways of
change may be affected by numerous different preconditions.
 As you develop your ToC, you will need to consider what assumptions are present and which ones
will affect the quality of your project.
Theory of Change
Pathways of
Change
Domains of
Change
Preconditions
Long-term
Change

18 August 2025 22
Improved IDP participation in
agricultural production and
income-generating activities
Improved or stabilized nutrition
among IDPs
Malnourished women and
children have access to
supplementary and
therapeutic support
Income-generating
activities among IDPs
increased
Agricultural production
among IDPs improved
Agricultural
user groups
established
Agricultural
use of water
increased
Access to
loans
increased
Marketing
skills improved
Nutrition
surveillance system
established
Food distribution
centers
operational
Infant and young
child feeding
practices improved
Increased knowledge
among mothers’
group participants
Feeding practices
improved
Decreased incidence of
waterborne disease among IDPs
IDPs have improved
access to adequate water
supply
Hygiene practices
improved
Chlorination
procedures
operational
Water points
established
WASH
volunteers
trained
Handwashing
campaign in
place
Improved livelihoods of internally
displaced persons (IDPs) in the Delta
River Region
Host
communities
receptive
Security
doesn’t
worsen
Legal status
for IDP
businesses
achieved
Flooding and
drought do
not exceed
10-year
average
Government
meets latrine and
water system
obligations
Inflation
remains
stable
IDPs have
access to
land and
water

18 August 2025 23
A results framework (RF) is the next step after a ToC to translate its contents. A results
framework is a logical model that organizes the results of a project into a series of if-then
relationships. The statements in the RF articulate the project’s hierarchy of objectives,
describing the causal (or vertical) logic of the project.
 A ToC and RF are different in one important way: while the ToC is a ‘big picture’ document
that identifies preconditions to achieve long-term change, a RF only includes
interventions that are the direct responsibility of the project team.
Results Framework

18 August 2025 24
Criteria for determining what is included in
project intervention
Strategic
considerations:
• What are the
strategic priorities
for your
organization in
the region?
Country? Other?
Technical feasibility
and sustainability:
• Can the proposed
work be realistically
accomplished?
• Can the work be
sustained and
maintained?
Institutional
capacity:
• What are your
organization’s
strengths and
weaknesses?
• What are your
implementing partners’
capacity levels?
Portfolio
considerations:
• Does the project
fit within the
larger portfolio of
projects in your
organization?
Resource availability:
• Is funding available?
• Is there potential for
growth?
• What opportunities exist
to obtain additional
resources?
Appropriateness:
• Is the proposed
approach acceptable to
the target population and
key stakeholder groups?
For example, would a
reproductive health
program be appropriate
and consistent with
religious and cultural
norms?
Financial /
economic
feasibility:
• Is the project
investment justified
based on the
anticipated return?
External program
considerations:
• Who else is working
in the proposed area
of intervention? What
are their program
strengths?
Needs prioritization:
• Which needs received the
highest level of emphasis during
the assessment/analysis?
• Addressing which needs would
appear to have the highest
potential for impact?
• Who stands to benefit the
most?
• How will the different needs
relating to gender, age and
socioeconomic status be
accommodated?

18 August 2025 25
Some RF models use four levels, which include a hierarchy of objectives comprised of a goal,
strategic objectives, intermediate results, and outputs. Other RF models may not follow these
levels specifically.
 To translate the ToC to a RF, the long-term change corresponds to the goal, the domains
of change correspond to the strategic objectives, and preconditions correspond to the
intermediate results and outputs.
Four-level hierarchy of objectives
Goal
•The goal describes the longer-
term, wider development to
which the project contributes.
Goal statements are usually
aspirational, focusing on states
of sustainability, livelihood, well-
being, etc.
Strategic
Objectives
•The SOs express the central
purpose of the project. They
describe the significant
benefits that are anticipated
by the end of the project. In
most cases, the SOs address
the immediate causes of the
core problem.
Intermediate
Results
•The IRs express the
expected change(s)
in behaviors,
systems, policies or
institutions as a result
of project outputs and
activities.
Outputs
•Outputs are the deliverables
resulting from project activities.
They include products, goods,
services, knowledge, skills and
attitudes. (e.g., people trained
with increased knowledge and
skills; quality roads built). There
may be more than one output
for each IR.

18 August 2025 26
Translating the ToC Content into RF Objectives
Statements
Precondition Precondition Precondition Outputs
Intermediate
Results
Strategic
Objectives
Domain of Change
Precondition Precondition
Long-Term Change Goal
Assumption

18 August 2025 27
Case Study RF
Internally displaced people in the Delta
River Region have improved livelihoods
GOAL
STRATEGIC
OBJECTIVES
There is a reduced incidence of waterborne
disease among IDPs.
1. IDPs have increased
access to an adequate
water supply
2.1 Volunteers improve
knowledge of WASH
principles
1.1 Community water boards
establish water points in IDP
communities
2. IDPs improve their
handwashing practices
2.2 IDP communities
increase knowledge
of handwashing
OUTPUTS
INTERMEDIATE
RESULTS

18 August 2025 28
A logframe describe the key features of the project (objectives, indicators, measurement
methods, and assumptions), and highlights the logical linkages between them. The logframe
informs the MEAL plan.
 Unlike the ToC and RF, a logframe also includes indicators and measurement methods.
 Indicators are measures used to track progress, reflect change, or assess project
performance.
 Measurement methods identify how the project will gather the data to track the progress
of the indicators.
 The following is an outline of a five-level matrix structure for logframes.
Logical Framework (Logframe)

18 August 2025 29
Objective
Statements
Assumptions
Objective statements define
the ‘vertical logic’ of the
project.
Objective Statements
• A logframe includes an objective statement that was not included in the RF:
activities.
 An activity describes the work that will be conducted to deliver the
project outputs.
• Unlike the RF, activities are included in the logframe, which describe the
work that will be conducted to deliver the project outputs.
• At the higher levels of the Logframe (the goal and strategic objectives), the
objectives statements tend to be more strategic, and focus on articulating
the outcomes of the project. At the lower levels of the Logframe (outputs
and activities), the objectives statements tend to be more operational, and
focus on articulating the outputs of the project.
Logframe Four-Level Structure

18 August 2025 30
Assumptions complement the
‘vertical logic’ of the hierarchy
of objectives by introducing
the ‘horizontal logic’ of the
project.
Assumptions
Logframe Five-Level Structure
• Making assumptions explicit provide a reality check by pointing out that
vertical logic succeeds if and only if the assumptions at each level of the
Logframe hold true.
Objective Statements
• Goal
• Strategic Objectives
• Intermediate results
• Outputs
• Activities
Assumptions
• IF the strategic objectives are met and the assumptions hold true at
the strategic objectives level, THEN they should all contribute to the
goal.
• IF the intermediate results are produced and the assumptions hold
true at the intermediate results level, THEN the strategic objectives
can be met.
• IF the outputs are completed and the assumptions hold true at the
outputs level, THEN the intermediate results can be produced.
• IF the activities are conducted and the assumptions hold true at the
activities level, THEN the outputs can be completed.

18 August 2025 31
An indicator is a measure used to
track progress, reflect change or
assess project performance.
Making assumptions explicit provide
a reality check by pointing out that
vertical logic succeeds if and only if
the assumptions at each level of the
Logframe hold true.
Indicators
• To identify indicators, ask if you need data to:
 i) Comply with donor reporting requirements?
 ii) Appreciate the level of project progress and achievement?
 iii) Analyze any variance between expected and actual performance?

 iv) Understand how or why change is happening?
 v) Discuss results with community groups, government agencies or
organizations?

18 August 2025 32
Kahoot!
(Chapter 2)

18 August 2025 33
Goal
• In general, a project Logframe would not include indicators at this level. Goal-level indicators reflect longer-term impacts that
are usually not achieved through the completion of a single project.
Strategic
Objectives
• Indicators reflect change that is sought, often from a single initiative, among extended participants, target populations, and
partners.
Intermediate
Results
• Indicators reflect the expected change(s) in identifiable behaviors of a specific group or the expected change(s) in systems,
policies or institutions required to achieve the higher outcome.
Outputs
• Indicators represent tangible goods and services delivered by the initiative. Examples of output-level language include:
people trained with increased knowledge and skills, quality roads built, goods delivered, and services performed.
Activities
• Unless specifically mandated by your organization or a donor, Logframe templates do not typically develop indicators to track
completion of activities.
Indicator Description by Objective Statement Level

18 August 2025 34
 Slightly different terminologies, include indicators since we have eliminated the
programme LFA template to minimise mandatory documents for HQ
Country Programme Results Framework
 DCA also has a Global Results Framework
DCA-draft-summaryglobalresults_03-06-2022 (shared with the board) (1).docx
DCA Results Framework tempate

18 August 2025 35
SMART Indicator Checklist
Indicators must be very closely related to the desired areas of improvement expressed in the
objectives statements that they represent, and should include the following characteristics:
Quantity The expected numerical representations of what is to be achieved
Quality The expected achievements described using words and/or graphics
Location The geographic boundary of the expected achievements
Target population The person or people expected to make/experience the anticipated change.
Indicators should be written in a way that promotes an accurate assessment of progress.
Indicators must be attainable given the budget, time and resources available.
Indicators must accurately measure the change you want to track.
Indicators must identify the timeframe within which the change is expected to occur.
Specific
Measurable
Achievable
Relevant
Time-bound
Once you are clear about the information you need, you can begin to identify your Logframe
indicators.
• As you identify indicators, you can use the SMART indicator checklist below to determine whether they
meet quality standards.
• SMART is an acronym that identifies five criteria – specific, measurable, achievable, relevant, and time-
bound - that together help teams assess the quality of project indicators.

18 August 2025 36
To understand the components of strong indicators, let’s examine two indicators that were created
for the Delta River IDP Project. Each indicator is written to be specific, measurable, and time-bound.
Case Study: Delta River IDP Indicators
Indicator Statements
What is measured
Target population
Unit of measurement
Direction, size or magnitude of the
change
Time frame
Each quarter, 100 percent of water points
managed by community water boards meet
WHO water quality standards
Water quality
Water points managed by community
water boards
Percentage
100 percent
Quarterly
By Year 3 of the project, 80 percent of IDPs
demonstrate knowledge that hands need to
be washed with soap after critical events
Knowledge that hands need to be
washed with soap after critical events
IDPs
Percentage
80 percent
By year 3 of the project

18 August 2025 37
Question whether there are standard or validated indicators that can be reused or repurposed
for your needs. Using standard indicators often increase indicator quality, follow donor
requirements, and help organize data across the organization and sector.
 Standard indicators are recommended and preferred whenever possible, especially for
higher-level objectives. However, there will be occasions when standard indicators are not
available or do not meet your specific information needs. In those cases, you will need to
develop a custom indicator.
Standard or custom indicators

18 August 2025 38
Direct indicators track change by directly examining what you are trying to measure.
 However, change may not be directly measurable, and you will need to identify indirect indicators
that approximate change in the absence of a direct measure. Indirect or proxy indicators track
change by examining markers that are generally accepted as being proxies for what you are
trying to measure. Proxy indicators are especially helpful when the result you are attempting to
monitor is difficult or too expensive to measure.
Direct or indirect / proxy indicators?
Indicator Example
Direct indicator: By Year 3 of the
project, 80% of IDPs increase
handwashing at critical times.
Proxy indicator: Both soap and water
are consistently present at latrine
locations.
Advantages
● The indicator attempts to directly
assess handwashing behavior.
● Research shows that the presence of
soap and water is associated with
increased handwashing.
● Collecting this proxy data is easier and
less expensive than direct observation of
handwashing.
Disadvantages
Collecting data requires:
● More time and budget
● Skilled observers
Observing handwashing could change
the behavior of community members.
Cannot reveal the frequency,
consistency or quality of handwashing
by individuals.

18 August 2025 39
You need to decide whether you require a quantitative or qualitative indicator to measure
progress toward your objective statements.
 Quantitative indicators: measures of quantities or amounts. They help you measure project
progress in the form of numerical information, such as numbers, percentages, rates, and
ratios.
 An example of a quantitative indicator from the Delta River IDP Project Logframe is “By Year
3 of the project, 85% of IDP households are located no more than 500 meters from a water
point”.
 Qualitative indicators: measure judgements, opinions, percentages, and attitudes toward a
given situation or subject. This indicator is often more subjective.
 An example may be “Female IDPs feel safe collecting water from IDP water points”.
Quantitative or qualitative indicators?

18 August 2025 40
They identify how the project will
gather the data to track the
indicators. Measurement
methods can be divided into two
categories: quantitative and
qualitative.
Measurement methods
• Quantitative methods: collect data that can be counted and subjected to
statistical analysis.
 They measure quantities, such as pure numbers, ratios, or
percentages.
• Qualitative methods: capture participants’ experiences using words,
pictures, and stories.
 It is collected through prompting questions that trigger reflection, ideas,
and discussions. Qualitative data are analyzed by identifying themes,
topics, or keywords.
• Each approach has various strengths and weaknesses.

18 August 2025 41
Comparing quantitative and qualitative measurement methods
Strengths Weaknesses
Quantitative
methods
Scalable: Processing results from a larger number of
subjects
Generalizable: Using data gathered from a sample,
assumptions can be made about patterns in the general
population
Objective: There is less personal bias in the collection and
analysis of data
Standardized: Data collectors use standard approaches
whose results can be compared to other data
Suited to ICT4D: Well-suited to use of digital devices for
data gathering and analysis
● Results from quantitative methods
sometimes miss the depth and complexity
of an issue
● Not suitable for identifying and exploring
unanticipated or unexpected factors
Qualitative
methods
Provide depth and detail: Provide detailed descriptions of
situation, providing a rich context
Create openness: Encourage people to expand on their
responses and potentially open up new areas of inquiry
Simulate people’s individual experiences: Provide a
detailed picture of why people act in certain ways and the
feelings behind these actions
Identify the unexpected: Helpful for identifying and
exploring unanticipated or unexpected factors
● Results from qualitative methods are
harder to generalize to a larger population
● Data are relatively difficult to collect and
analyze
● Data are susceptible to the hidden bias of
collectors and participants
● More difficult to transcribe data directly to
digital devices

18 August 2025 42
 Using either quantitative or qualitative measures alone may be insufficient for tracking and
understanding change. MEAL practitioners often advocate for a mixed approach that employs
both types of measurement methods.
 Mixed methods approach: deepens understanding of the project, provides more
comprehensive integrated data for tracking progress, analyzing results, and making decisions.
A mixed-methods approach can strengthen your data, analysis, and interpretation if you
consciously incorporate triangulation.
 Triangulation is the validation of data through cross-verification of more than two sources.
Using triangulation, you can cross check and reinforce results.
Mixed-methods approach

18 August 2025 43
The project used mixed methods to gather different types of information:
 For the indicator “By Year 3 of the project, 80% of IDPs self-report increased
handwashing at critical times”, UNITAS used both questionnaires and focus
groups to collect information.
Case Study Mixed Methods
Questionnaire
(quantitative)
• Has handwashing behavior increased after critical events?
• Has knowledge of handwashing behavior increased?
Focus group(s)
(qualitative)
• Which project activities were particularly effective at changing handwashing behavior?
• Are there factors preventing the target population from adopting handwashing behavior?
• Is improved handwashing behavior resulting in valuable change?

44
Case Study: Logframe
Objective Statements Indicators
Measurement
Methods
Assumptions
Goal: IDPs in the Delta River Region have improved livelihoods
Strategic objective: There
is reduced incidence of
waterborne disease among
IDPs.
● Incidence of waterborne disease among IDPs is reduced by 30 percent
by the end of Year 3.
● Municipal hospital
and clinic records
collected by mobile
health teams
● No other sources of waterborne disease
increase significantly
● IDPs sustain adoption of improved hygiene
practices
Intermediate result 1: IDPs
have improved access to
adequate water supply
● By Year 3, 75 percent of IDPs indicate water access meets their
household consumption needs
● By Year 3, 85 percent of IDP households are located no more than 500
meters from a water point.
● Each quarter, 100 percent of water points managed by community
water boards meet WHO water quality standards
● By Year 3, an average of 30 liters of water per person per day is
available to IDPs through water points
Intermediate Result 2: IDPs
improve their handwashing
practices
● By Year 3, 80 percent of IDPs self-report increased handwashing at
critical times
● Both soap and water are consistently present at latrine locations
Output 1.1: Community
water boards establish water
points in IDP communities
Output 2.1: Volunteers
improve knowledge of WASH
principles
Output 2.2: IDP
communities increase
knowledge of handwashing
● By Year 3, 80 percent of IDPs demonstrate knowledge that hands need
to be washed with soap after critical events
● By Year 3, 75 percent of IDP women indicate higher levels of
confidence preventing waterborne disease
● By Year 3, 40 water points are established (4 per village)
● By Year 3, 10 trained community water boards are operational
● By Year 3, 100 percent of water points meet water quality standards
● 100 percent of WASH volunteers can effectively explain WASH
principles to IDPs upon completion of training events
● 40 WASH volunteers pass the certification exam each year
● Questionnaire
● Focus group(s)
● Latrine visits
● Quarterly water
board reports
● Training workshop
attendance sheets
● Certification results
● Questionnaire
● Focus group(s)
IDP cultural and religious customs are not
violated by the behavior change messages.
● WASH volunteers are trusted by IDPs
● Spare parts and trained mechanics are
available in the event of water point breakdowns
● Water points are established in locations that
are safe and secure for women and children
● Questionnaire
● Focus group(s)
● Water point visits
● Quarterly water board
reports
● Government meets its water system obligations
● IDPs do not obtain water from sources other
than the designated water points
Activities IR 1: 1.1.1: UNITAS and water board identify new water point locations; 1.1.2: UNITAS sources and distributes water point materials; 1.1.3: Community water boards identified,
trained and functioning; 1.1.4: UNITAS team develops water board quality report format.
Activities IR 2: 2.1.1: UNITAS sanitation team identifies WASH volunteers and trainers; 2.1.2: Sanitation team designs WASH training curricula and materials and identifies training
locations and times; 2.1.3: Sanitation team delivers WASH training to volunteers; 2.1.4: Sanitation team develops handwashing campaign materials/mechanisms; 2.1.5: Community
Objective Statements Indicators Measurement Methods Assumptions
Goal: IDPs in the Delta River Region have improved livelihoods
Strategic objective: There is
reduced incidence of waterborne
disease among IDPs.
● Incidence of waterborne disease among IDPs is reduced by 30 percent by
the end of Year 3.
● Municipal hospital and
clinic records collected by
mobile health teams
Intermediate result 1: IDPs have
improved access to adequate water
supply
● By Year 3, 75 percent of IDPs indicate water access meets their household
consumption needs
● By Year 3, 85 percent of IDP households are located no more than 500
● Each quarter, 100 percent of water points managed by community water
boards meet WHO water quality standards
● By Year 3, an average of 30 liters of water per person per day is available to
IDPs through water points
● Questionnaire
● Focus group(s)
● Water point visits
reports
● Government meets its water system obligations
● IDPs do not obtain water from sources other than
the designated water points
Intermediate Result 2: IDPs improve
their handwashing practices
● By Year 3, 80 percent of IDPs self-report increased handwashing at critical
times
● Both soap and water are consistently present at latrine locations
● Questionnaire
● Focus group(s)
● Latrine visits
● No other sources of waterborne disease increase
significantly
● IDPs sustain adoption of improved hygiene
practices
Output 1.1: Community water boards
establish water points in IDP
communities
● By Year 3, 40 water points are established (4 per village)
● By Year 3, 10 trained community water boards are operational
● By Year 3, 100 percent of water points meet water quality standards
reports
● Spare parts and trained mechanics are available
in the event of water point breakdowns
● Water points are established in locations that are
safe and secure for women and children
Output 2.1: Volunteers improve
knowledge of WASH principles
● 100 percent of WASH volunteers can effectively explain WASH principles to
IDPs upon completion of training events
● 40 WASH volunteers pass the certification exam each year
● Training workshop
attendance sheets
● Certification results
● WASH volunteers are trusted by IDPs
Output 2.2: IDP communities
increase knowledge of handwashing
● By Year 3, 80 percent of IDPs demonstrate knowledge that hands need to
be washed with soap after critical events
● By Year 3, 75 percent of IDP women indicate higher levels of confidence
preventing waterborne disease
● Questionnaire
● Focus group(s)
IDP cultural and religious customs are not violated
by the behavior change messages.
Activities IR 1: 1.1.1: UNITAS and water board identify new water point locations; 1.1.2: UNITAS sources and distributes water point materials; 1.1.3: Community water boards identified, trained and
functioning; 1.1.4: UNITAS team develops water board quality report format.
Activities IR 2: 2.1.1: UNITAS sanitation team identifies WASH volunteers and trainers; 2.1.2: Sanitation team designs WASH training curricula and materials and identifies training locations and times;
2.1.3: Sanitation team delivers WASH training to volunteers; 2.1.4: Sanitation team develops handwashing campaign materials/mechanisms; 2.1.5: Community volunteers implement handwashing promotion
event

18 August 2025 45
Kahoot!
(Chapter 3)

18 August 2025 46
2. Planning MEAL Activities
Phase 1:
Designing
logic models
Phase 2:
Planning
MEAL
activities
Phase 3:
Collecting
MEAL data
Phase 4:
Analyzing
MEAL data
Phase 5: Using
MEAL data both
externally and
embed learning.

18 August 2025 47
 Identify and describe the purpose, process and content of key MEAL planning tools of Performance
management plan / MEAL plan
• Indicator Performance Tracking Table (IPTT)
• Feedback-and-response mechanism flowchart
• Learning plan
• Planning tools for MEAL communications
• Summary evaluation table
• Evaluation terms of reference
 Understand the various types of evaluation and the purpose of each
 Explain why MEAL planning is important and understand its relationship to broader project planning
and project management.
Objectives

18 August 2025 48
You will need to answer the question, “How will we collect, analyze, interpret,
use, and communicate MEAL information”.
MEAL Planning Tools
Planning tool
Performance management plan
(or a monitoring and evaluation
plan)
Indicator Performance Tracking
Table
Feedback-and-response
mechanism flowchart
Learning plan
Planning tools for MEAL
communications
Summary evaluation table
Evaluation terms of reference
Content
Builds on the Logframe, providing additional information on indicator definitions, data
collection plans, means of analysis, and data use.
Helps teams track progress toward a project’s indicator targets in an easy-to-read table
format.
Maps the flow of feedback from stakeholders and identifies how the project will respond
to the feedback it receives.
Ensures learning activities are intentionally planned and managed throughout the life of
the project.
Identifies stakeholder information needs and helps ensure that MEAL communications
are systematically planned and managed throughout the life of the project.
Describes planned evaluations, including priority questions, timing and budget.
Plans the specifics of an evaluation, including concise evaluation questions, proposed
methods, and roles and responsibilities.

18 August 2025 49
All projects should have a MEAL plan regardless of their size, complexity, or value. MEAL plans
tell you specifically what will be monitored and evaluated, and how these activities will take
place.
 MEAL plans should answer the following questions:
 How are the indicators defined?
 Who is responsible for MEAL activities?
 When will MEAL activities take place?
 How will data be analyzed?
 How will data be used?
 Included in the MEAL Plan are objective statements from the Logframe. However, it does not
include goal or activity statements as it is unlikely the project will track progress against these
statements.
Performance Management Plan / MEAL Plan

18 August 2025 50
Performance Management Template / MEAL
plan
Performance Management Plan
Objective
Statements
Indicators
(with
definitions as
needed)
Data Collection Means of Analysis Use of
information for
communicatio
n and decision
making
Method Frequency Person who
will collect
data
Respondents
(who will talk
to )
Type of
analysis
Subgroups
(strata)
Strategic
Objective 1
Strategic
Objective 2
Intermediate
Result 1.1
Intermediate
Result 1.2
Output 1.1
Output 2.1
Key assumptions
Assumption 1
Assumption 2

18 August 2025 51
 MEAL Plan template
DCA’s MEAL plan template

18 August 2025 52
 Methods: Measurement methods were identified when the Logframe was developed, so that information
could be pulled directly into the MEAL plan.
 Timing and frequency: You will now need to determine when and how often to collect data. The timing
and frequency of data collection will be based on different factors, including:
 Responsibility: Identify who is primarily responsible for collecting the data in question.

Data Collection
Resource
availability
Estimates of how
quickly change is
expected to occur
Seasonal
considerations
(this might include
planting and
harvest
calendars, school
schedules,
weather patterns,
and religious
holidays)
Donor reporting
requirements
Management and
decision-making
needs

18 August 2025 53
Kahoot!
(Chapter 4)

18 August 2025 54
 After the PMP, you will need a tool to track
the performance of your project by regularly
documenting progress against project
targets.
 Indicator Performance Tracking Table
(IPTT): it distills the project’s information into
a short concise table format. It shows where
the project stands with regard to its original
and revised indicators, and shows progress
achieved toward the indicator targets.
Indicator Performance Tracking Table
Provides a simple format to establish
indicator targets and track progress
against them over time
Improves accountability for tracking
and reporting project progress
Compare the project’s progress to other
projects inside (or outside of) the
organization
Compare actual versus expected
performance and think critically to
understand the evidence
A
n
I
P
T
T
i
s
i
m
p
o
r
t
a
n
t
b
e
c
a
u
s
e
i
t
:

18 August 2025 55
Two key components of the IPTT are baselines and targets.
 Baseline: The value of an indicator before the implementation of an activity, against which
subsequent progress can be assessed. Ideally, all project indicators should have baseline
data which is collected before implementation of an intervention.
 Target: the specific, planned level of change to be achieved during the life of the project.
Stated as a number of percentage. All SMART indicators will have a target figure
associated with them.
 Note: targets at different levels of the IPTT have different frequencies of data collection. At
the output level, the annual indicator targets are broken down by quarterly metrics as change
happens more quickly at the output level and should be monitored.
IPTT Continued

18 August 2025 56
Donors may have specific requirements about how to structure your IPTT. Below is a partially populated IPTT.
Remember to include all indicators for the project in the model.
Case Study
Indicator Baseline Year 1 Year 2 Year 3
Target Actual Variance Target Actual Variance Target Actual Variance
Strategic Objective: There is
reduced incidence of
waterborne disease among
IDPs
Incidence of waterborne
disease among IDPs is
reduced by 30% by end
of Year 3.
1,200 patients treated for
waterborne disease in
calendar year 0.
↓10% ↓20% ↓20%
have improved access to
adequate water supply.
By Year 3, 85% of IDP
households are located
no more than 500 meters
from a water point.
0% 20% 50% 85%
improve their handwashing
practices.
By Year 3, 80% of IDPs
self-report increased
handwashing at critical
times.
30% report washing
hands at critical times
50% 65% 80% 0
Output 1.1: Community water
boards establish water points in
IDP communities.
By Year 3, 40 water
points established (4 per
village)
0 10 Yr1
2-q1
2-q2
3-q3
4-q4
30 Yr2
5-q1
5-q1
5-q1
5-q1
40 Y3
2-q1
2-q2
3-q3
4-q4
Output 2.1: Volunteers
improve knowledge of WASH
principles.
40 WASH volunteers
pass the certification
exam each year
0 40 Yr1
10-q1
10-q2
10-q3
10-q4
40 Yr2
10-q1
10-q2
10-q3
10-q4
40 Yr3
10-q1
10-q2
10-q3
10-q4

18 August 2025 57
 IPTT Template
DCA’s IPTT template

18 August 2025 58
FRMs are two-way communication mechanisms designed specifically to gather and respond to
feedback from your project participants and other community stakeholders.
 To ensure their success, feedback must flow in two directions:
Feedback and response mechanisms (FRMs)
PROJECT TEAM
COMMUNITY
Processes and uses
feedback data
Hotlines
Help Desks
Suggestion Boxes
Text messages
Radio programs
Meetings
 Feedback mechanisms: Communities provide feedback
to the project team through channels that include
meetings, suggestion boxes, hotlines etc.
 Response mechanisms: The project team acknowledges
receipt of the feedback and provides appropriate
responses to the community.
 A project should determine how feedback will be
collected, who receives feedback, how feedback is
processed, how feedback is used, and how the project
will respond to feedback.

18 August 2025 59
When designing an FRM for your project, be sure to:
 Once the FRM design is complete and procedures are documented, create specific, clear
instructions on how communities can access and use the FRM.
Designing FRMs
a. Respond to all feedback
received. In some cases,
this will only require an
acknowledgement of
receipt, in other cases the
response could be ongoing
and complex.
b. Ensure that response
mechanisms are
appropriate to your project
context and the type of
feedback received.
c. Identify an appeals
process in case you receive
a second round of feedback
from the community
indicating that additional
follow-up is required.

18 August 2025 60
Isabella Hjorth-Falsted, Accountability and Complaints Advisor
ihfa@dca.dk
FRMs Focal point in DCA

18 August 2025 61
For projects to be effective, teams must adapt in response to changing contexts and new
information.
 To focus on project learning, adaptive management is an intentional approach to making
decisions and adjustments to the project in response to new information and changes in
context.
 Include learning-to-action discussions (LADs) as part of project activities to practice
adaptive management. They are planned discussions that bring staff together to reflect on
data, project progress, and future directions. They occur during the data collection
process.
 The information generated from project MEAL systems can then be used for larger
organization learning, which is the process that the organization discovers and adapts to new
knowledge. Knowledge creation, knowledge transfer, and knowledge retention are three
concepts that contribute to organizational learning.
Learning Plan

18 August 2025 62
Documenting Learning
Timeline:
The key milestones
and deadlines for the
activity or process.
This timeline should
be linked to project
implementation
calendars.
Roles and
responsibilities:
The roles and
responsibilities of the
office or staff member
responsible for
leading the activity or
process.
Activity or
process:
A concise
description of the
specific activity or
process.
Resources:
The resources
(including staff time,
mechanisms,
implementing partners,
funding, etc.) needed
to implement the
action item.
Expected outcomes:
The intended outcome
for each action item
describing the
anticipated changes
resulting from
implementing the
identified activity or
process.
Your learning plan should include:

18 August 2025 63
Ideally, all projects will create communication plans to bring greater intentionality to meet
stakeholder information needs.
 A communication plan defines who needs to be aware of and informed about the project
MEAL activities, what they need to know, how and how often information will be distributed,
and who will be responsible for the distribution. Generally, it should include the:
MEAL Communication Plans
Target stakeholder
• Stakeholders have different information needs and different communication preferences.
Information needs
• What does each audience need to know? This may include stakeholder information needs on project goals and objectives,
access to and use of feedback-and-response mechanisms, project progress, changes and updates, and results of learning
efforts.
Communications methods
• Information should be as accessible as possible with stakeholder preferences in mind.
Timing and frequency
• Often, different stakeholders require information at different times.

 A single communication method will not work for all stakeholders. When considering what methods to
use, the project should look at literacy rates, the size of the audience, and whether the communication
will be one-way or a two-way exchange. Ensure that your communications are transparent,
participatory, and responsive.
18 August 2025 64
Communication Methods
BROAD
AUDIENCE
TARGETED
AUDIENCE
HIGH LITERACY
LOW LITERACY
Radio
Newspaper
articles
Leaflets
and flyers
Notice
boards
Project
reports
One-way Communication
Conferences
Workshops
Community
meetings
Hotlines and
help desks
Door-to-
door visits
Mobile
phones (text
and voice)
Social
media
Two-way Communication

18 August 2025 65
All projects should include an evaluation activity. The type of evaluation you do and the timing
of your evaluation(s) will be dictated by your information needs.
Evaluation Planning
Type Purpose Timing
Formative Improve and refine an existing project Early in project implementation, up to the midpoint.
Process Understand how well a project is being implemented (or was implemented) particularly
if you want to replicate or enlarge your response.
During project implementation (often at the midpoint) or at the end.
Impact or
outcome
Assess how well a project met its goal to produce change. Impact evaluations can use
rigorous data collection and analysis, and control groups.
At project end. Also requires baseline data gathered at the beginning
of implementation and regular, rigorous monitoring activities.
Summative Judge the performance of the project. At project end.
Ex-post Assess the long-term sustainability of the project. After the project’s formal end date, sometimes 3 to 5 years later.
Developmental
evaluation
Used to design a response to a known need, particularly in complex situations, where
response approaches are being tested. It supports creative, innovative approaches
and provides real-time feedback to inform ongoing project design.
Continually throughout project implementation.
Empowerment
evaluation
An approach that seeks to improve project implementation by providing project
participants themselves with the tools to evaluate the planning, design and
implementation of the project.
Throughout project implementation in the sense that participants
require training and facilitation in evaluation tools. The evaluation
becomes part of project implementation.
Meta evaluation A systematic and formal evaluation of evaluations. Examines the methods used within
an evaluation or set of evaluations to bolster the credibility of findings. Often used in
policy-making settings.
External to project implementation cycle.

18 August 2025 66
A summary evaluation table should be completed at the beginning of your project and brings
out the details and purpose of the evaluations your project intends to conduct. Included in this
table are the:
 Evaluation questions are clear statements of what you need to know from the evaluation.
They will vary depending on the evaluation criteria you explore.
 To establish your questions, a preliminary step is identifying evaluation criteria. Common
criteria are relevance, efficiency, effectiveness, impact, and sustainability.
Summary Evaluation Table
Evaluation
purpose
Priority
evaluation
questions
Timing
Anticipated
evaluation
start and
completion
Evaluation
budget

18 August 2025 67
The ToR should explain the project, your evaluation purpose, evaluation questions, and the
methods you suggest for collecting data. An evaluation ToR should include the following:
Evaluation Terms of Reference
Project
introduction and
background:
Briefly describe
the project, its
implementation
period, funding
sources and
amounts, and any
other relevant
information.
Summarize the
project, the
problem that it
seeks to solve, its
intervention
strategy, and
what baseline
and monitoring
data exists.
Evaluation
purpose,
audience, and
use:
Explains why you
conduct the
evaluation, who
will use it, and
how it will be
used.
Evaluation
criteria and
questions:
Review the
evaluation criteria
and questions in
the summary
evaluation table.
Include only
questions that
you need to
answer.
Methodological
approach:
Include a brief
description of a
suggested
approach that
incorporates the
monitoring
processes and
data already in
place. Any
methods chosen
must be linked to
the questions that
need to be
answered.
Evaluation roles
and
responsibilities:
Describe the
various roles and
responsibilities of
the evaluation
team. Include
detail about data
collection, data
analysis, and
report-production
responsibilities.
Evaluation
deliverables and
timeline:
State the timeline
of the evaluation
and when
different
components are
due. Allocate time
for document
desk review,
fieldwork, data
analysis, report
writing, and
stakeholder
feedback and
response.
Evaluation
logistics and
other support:
Include details
about logistics
and support to
the evaluation
team.

18 August 2025 68
When planning for MEAL, it is important to develop budgets and calendars that are
comprehensive and offer a detailed list of all project MEAL activities.
 MEAL in the project calendar: all MEAL activities in the planning documents should be listed,
including all monitoring visits, evaluation activities, learning initiatives, feedback-and-
response mechanisms, communications efforts, and any reports that need to be created.
Using this information, build a Gantt chart for the MEAL activities.
 A Gantt chart is a bar chart that illustrates a project schedule, identifying the start date, end
date and expected duration of all activities.
 The next slide is an exemplar of a Gantt chart for the first 3 years of MEAL activities for
the Delta River IDP Project. Activities are taken from the performance management
plan, the summary evaluation table, and the communications plan for the project.
MEAL in Project Management

18 August 2025 69
Year 1 Year 2 Year 3
1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 1
Monitoring Activities
FRM meetings
Quarterly water board visits
Project questionnaire design
Focus group discussion design
Enumerator training
Annual questionnaire conducted
Annual focus group conducted
Data analysis and interpretation
Community updates
Annual donor report
Quarterly water board reports
Evaluation activities
Project midterm evaluation
Project after action review

18 August 2025 70
After your project proposal is approved, a more detailed budget needs to be
created.
 Detail budgets are often activity based, meaning that they are often found in
MEAL planning documents and the MEAL Gantt chart.
MEAL in the Project Budget

18 August 2025 71
Kahoot!
(Chapter 5)

18 August 2025 72
3. Collecting MEAL Data
Phase 1:
Designing
logic models
Phase 2:
Planning
MEAL
activities
Phase 3:
Collecting
MEAL data
Phase 4:
Analyzing
MEAL data
Phase 5: Using
MEAL data both
externally and
embed learning.

18 August 2025 73
 Explain the five elements of data quality
 Describe the components of a basic data collection tool outline
 Identify three primary methods of data collection and key characteristics of each
(questionnaires, interviews and focus group discussions)
 Explain the basic principles of sampling
 Describe key steps in preparing to implement data collection tools
 Identify generally accepted protocols and standards for responsible data management
 Understand the basics of selecting databases and associated data entry and cleaning
practices
Objectives

18 August 2025 74
Data can never be free of bias. Thus, you need to determine what quality and quantity of data
is sufficient for your decision-making, learning, and accountability needs. When thinking about
data quality, consider the following five data quality standards:
Data Quality
Validity
Data are valid
when they
accurately
represent what
you intend to
measure (the
indicators). Make
sure your
collection methods
will collect the
data you want to
measure your
indicators.
Reliability
Data are reliable
when the
collection methods
used are stable
and consistent.
Reliable data are
collected by using
tools such as
questionnaires
that can be
implemented at
different times in
the same way.
Precision
Data are precise
when they have a
level of detail that
gives you an
accurate picture of
what is happening
and enables you
to make good
decisions. Ensure
that precise data
are collected
using appropriate
sampling
methods.
Integrity
Data have
integrity when
they are accurate.
Data should be
free of the kind of
errors that occur,
consciously or
unconsciously,
when people
collect and
manage data.
Timelines
Timely data
should be
available when
you need it for
learning that
informs decisions
and for
communication
processes.

18 August 2025 75
When developing your data collection tools, revisit the question “What do I need to know?”
 All data collection tools are designed using a similar outline. Walking through this outline
helps illustrate good practice for designing data gathering tools:
 Introduction: Gives you the chance to explain the project and the data collection process to the
respondent. This should include why information is being collected, how participants were identified, how
the data will be collected, how much time the data collection will take, how the data will be used, and
who will have access to the data.
 It should also explain the ethical principles that guide your data collection efforts. All tools should
include the principle of informed consent, specific plans to keep participant contributions confidential,
potential plans for compensating participation, and plans to share results with participants.
Developing Data Collection Tools

18 August 2025 76
After the introduction, your tool should list the questions to be asked of the respondent that
are designed to gather the data you need to meet your information requirements.
 When doing so, ensure that:
 When concluding, always offer the respondents a chance to ask questions and provide
feedback.
Data Collection Questions
the language you use in your questions is simple, clear, and free of jargon,
organize questions using a clear, orderly sequence
make sure that your data collection tool includes fields to record important
data analysis and management information such as the data and location of
data collection and participant identification.

18 August 2025 77
A questionnaire is a structured set of questions designed to elicit specific information from
respondents. Questionnaires primarily use closed-ended questions to generate responses that
are easy to code and analyze.
 Close-ended questions: questions that provide a predefined list of answer options. This
makes it easier for responses to be coded numerically and allow for statistical analysis.
 Consistent implementation is key to a successful questionnaire: the same questions are asked
of each respondent, in the same format, and order. This helps ensure that responses are
clear, valid, and reliable.
 Include options such as ‘I do not know’ and ‘Other’, all appropriate responses, and allow
respondents to ‘skip’ questions depending on their answer to the previous question.
 Also think about which media will be used to present the questionnaire and how responses will
be recorded.
Qualitative data collection tool: Questionnaire

18 August 2025 78
Types of close-ended questions
Question Type Example
Question Response Example
Numerical 1. “How long have you been displaced?” ____ number of months, or
❏ I don’t know.
Two-option response 2. “Are there handwashing facilities at the latrine?”
If no, skip Question 3 below
❏ Yes
❏ No
Multiple choice 3. “Which handwashing resources are currently
available at the latrine?”
❏ Water and soap
❏ Water only
❏ Soap only
❏ Neither water nor soap
❏ Other ____________
❏ I don’t know
Rating or Likert scale 4. Indicate the extent to which you agree with this
statement: “My household has enough water to
meet our household consumption needs.”
❏ Strongly disagree
❏ Disagree
❏ Neither agree nor disagree
❏ Agree
❏ Strongly agree

18 August 2025 79
Advantages and disadvantages of questionnaires
Advantages Disadvantages
Personal
interview
● Respondents don’t
need to be literate
● Facilitators can
motivate and support
respondents
● There is a high rate of
cooperation and a low
rate of refusal
● Activities are time-
consuming and expensive
● Facilitators can influence
respondents’ interpretation
of questions (and their
responses)
● Data entry can be
difficult if responses are
not collected using digital
devices
● Space and privacy
for interviews
● Budget for travel
● Trained facilitators
Self-
administered
questionnaire
● Easy and cheap to
distribute
● Access to a broader
population in a larger
geographic area
● Requires respondent
literacy
● Data input can be
cumbersome if responses
are not collected using
digital devices
● Potentially low response
rates
● Logistics for
distributing and
collecting
questionnaires
● Budget for distribution
and collection of
questionnaires
Requirements

18 August 2025 80
Qualitative data collection tools such as semi-structured interviews and focus groups are
designed to explore and understand the respondent’s perspectives, opinions, and ideas.
These closely resemble conversations.
Semi-structured interviews: A guided discussion between an interviewer and a
single respondent designed to explore and understand the rich depth and context of the
respondent’s perspectives, opinions and ideas.
Focus groups: A guided discussion between respondents in a group. It is a
qualitative data collection tool designed to explore and understand the rich depth and
context of a group’s perspectives, opinions, and ideas.
Qualitative data collection tools: Semi-
structured interviews and focus groups

18 August 2025 81
A focus group discussion provides an opportunity for a small group to interact with
each other and present different opinions. However, a focus group requires more
experience to facilitate.
Focus groups
Open-ended
questions
Recruit the right
participants: who
speak directly to
the perspectives
or experiences
you are interested
in.
Carefully plan
the questions
to frame the
conversation
 Guide discussions using open ended questions,
which allow someone to give a free-form response in
their own words. There are two types of open-ended
questions:
 Content-mapping questions, which initiate the
exploration of a topic by raising and broadly
exploring an issue.
 Content-mining questions, which are probing
questions that elicit more detail or explanation
about a response to a content-mapping question.

18 August 2025 82
UNITAS will be conducting focus group discussions to understand IDP access to potable
water. The focus group discussion guide includes this content-mapping question: “What are
the major barriers IDPs face when accessing potable water?”
 It is expected that some responses will require follow-up, so content-mining questions will
be useful. A potential content mining example is shown below.
Case study
“Can you tell me
more about why you
said, ‘I don’t feel
safe?” and “Can you
provide an
example?”.
“I don’t feel
safe”

18 August 2025 83
Appropriate sampling methods collect the right amount of data from the right respondents to
meet your information needs.
 Sample: a subset of the population or community that you choose to study that will help you
understand the population or community as a whole. It is necessary because gathering data
is expensive and time-consuming, making it difficult to speak to everyone and can be divided
into two groups, random sampling and purposive sampling. Purposive sampling will be
discussed later.
 Random sampling is used alongside quantitative methods. It is a probability sample that
includes respondents selected from a list of the entire population of interest so that each
respondent has an equal chance of being selected. Random samples are created using
mathematical calculations to identify how many people will participate in your data gathering
efforts.
Creating samples

18 August 2025 84
 Sample bias: occurs when some members of the population are more or less likely to be
selected for participation in your data gathering efforts than others. When your sample is
biased, you do not consider all available perspectives. Your data will not be valid or
accurate and cannot be easily generalized to the population you want to address.
18 August 2025 84
Sample bias
Voluntary response bias
occurs when data are
collected disproportionately
from self-selected
volunteers. This may exclude
people who are less
accessible and may also
over-represent people with
strong opinions related to the
project.
Convenient sampling bias
occurs when data are
collected from respondents
who are easy to reach, or
who are easy to work with.
Data that suffer from
convenience may over-
represent people in certain
areas and not be accurately
representative.
 When trying to reduce
sampling bias, pay attention
to two specific types of bias:
convenient sampling bias
and voluntary response bias.

18 August 2025 85
Define your population and sampling unit
 First, a population is a set of similar people, items or events that is of interest for some
question or experiment.
 When defining your population, state your inclusion and exclusion criteria. These may
include participation in project activities, geographic boundaries, or demographic
characteristics.
 After you are clear about your population criteria, you need to identify your sampling unit.
A sampling unit is the individual person, category of people, or object from whom/which
the measurement (observation) is taken. Examples may be children under 5 years of age,
adolescents, women, men, or households.
18 August 2025 85
How to identify a random sample
Step 1

18 August 2025 86
Choosing a method to calculate your random sample
 To calculate your sample, there are different random sampling methods you can use.
18 August 2025 86
 If your data analysis plans include disaggregation by subgroup, your sampling method should
specifically include those subgroups.
 Stratified sampling is a strategy that divides the population into separate subgroups (strata). Then, a
probability sample is drawn from each subgroup to allow for statistical comparison. If creating stratifying
samples, refresh your known on random sampling methods.
Simple Random Sample:
Every unit in your population
has an equal chance of being
selected.
Cluster Sampling:
The population is divided into
naturally occurring clusters
such as geographical areas,
schools or places of
employment. All clusters are
listed, and clusters are
randomly selected.
Systematic Sample:
A process of listing and
numbering all potential
subjects and then selecting
every 10th person, for
example, until you have
reached your sample size.
Step 2

18 August 2025 87
Determining your sample size
 To gauge how representative your sample is, you can analyze margin of error and
confidence levels.
 Margin of error expresses the maximum expected difference between the true population
and the sample estimate. To be meaningful, the margin of error should be qualified by a
probability statement (often expressed in the form of a confidence level).
 Confidence level refers to the percentage of all possible samples that can be expected to
include the true population parameter.
18 August 2025 87
 If your data analysis plans include disaggregation by subgroup, your sampling method should
specifically include those subgroups.
Step 3

18 August 2025 88
From the Delta River IDP Project, their questionnaire cites that 50% of respondents report
“washing their hands after critical events”. The confidence level of the survey is 95%, with a
margin of error of plus or minus 3%. This information means that the survey was conducted
100 times and that the percentage of reporting “washing their hands after critical events”
ranged from 47% to 53% most of the time.
 However, sampling can also affect your margin of error and confidence levels. For example,
a very small sample of 50 respondents has about 14% margin of error while a sample of
1,000 has a margin of error of 3%.
 To obtain a 3% margin of error at a 90% level of confidence, your sample size needs to be
around 750 in this population. While a 95% level of confidence requires a sample size of
around 1,000.
 Sample Size Calculator
18 August 2025 88
Margin of error and confidence level in practice

18 August 2025 89
Selecting your sample units
 When selecting your sample units, a sample frame is useful. It is a specific list of units
(men, women, households etc) that you will use to generate your sample. This could be a
census list, registration log etc.
 If you do not have a sample frame, you can use an alternative sampling selection
approach, such as a random route method. This is a systematic sample that can be used
when you don’t have a list of the total population. It requires you to sketch a map of the
community, estimate the total number of households, and calculate the number of
households to be included in the sample. Then, generate a random route through the
community by selecting a starting point on the map and instructing the interviewer which
path they should go on.
18 August 2025 89
Step 4

18 August 2025 90
 Purposive sampling is a non-probability sample where sampling units that are investigated
are based on the judgement of the researcher. Sampling units are selected based on
characteristics of a population and the objective of the study. They are used primarily when
you want to collect qualitative data. Your sample units are deliberately, rather than
randomly selected to reflect important features of groups within the sample population.
 It highlights the experience or perspective of a particular group by offering a “deep”
understanding at the level of the individual participant.
18 August 2025 90
Purposive Sampling

18 August 2025 91
Step 1 in identifying a purposive sample: identify the type of purposive sampling to use.
 Start by defining your population and sample frame. Establish sampling criteria that clearly
define the sampling units you will use. Next, select the sampling method you will use to
identify your purposive sampling. There are different methods of purposive sampling.
18 August 2025 91
Purposive Sampling
Best and worst-case
sampling
Compares
communities or
individuals who are
considered the best
and worst cases
based on certain
characteristics (i.e.,
most vulnerable and
least vulnerable).
Typical case sampling
Provides an
understanding of the
general scenario by
choosing those
communities or
individuals who are
considered average.
Critical case sampling
Collects information
from communities or
individuals who are
important for
understanding a
particular context or
situation.
Quota sampling
Attempts to collect
information from
participants with
characteristics of
interest according to
estimates of their
proportion in the
population.
Snowball or chain
sampling
Collects information from
participants in stages,
starting with respondents
known to the evaluators
or partners and then
asking those
respondents for
recommendations of who
else to speak to. This
method helps you
identify sources of
information previously
unknown to you.

18 August 2025 92
Determine your sample size
 Sample size is calculated differently in purposive sampling compared to random sampling.
Often, qualitative data is used to triangulate or cross check quantitative data. As a result,
you will need to conduct enough interviews or focus groups to test, reinforce, and confirm
the patterns that are emerging.
 Purposive sampling sizes are decided based on two factors:
18 August 2025 92
Purposive Sampling
For example, if you use the best-
and worst-case purposive
sampling method to conduct focus
group discussions on women’s
opinions about access to water
points, plan to conduct at least two
or three focus group discussions
to collect information from each
perspective (best case and worst
case).
1) If the data analysis plan in your PMP requires that you
compare subgroups, you will require a large sample, and
the size increases exponentially the more subgroups
you have.
2) Budget constraints and resource limitations influence
your sample size decisions.
Step 1

18 August 2025 93
Before you can begin to collect your data, there are several steps that you should follow.
Translate your data collection tools: is your project working in a region that uses
multiple languages? If so, the tool will
need to be translated to mitigate language bias.
Train data collectors and test your tools. Training should include:
Using data collection tools
Step 1
Step 2
 The basic ethical principles of good data collection
 The purpose of the tool, the purpose of each question, and how the answers will be analyzed and used.
 Instruction that emphasizes the skills needed to use the tool. When collecting quantitative data, enumerators
need to know the order of questions to ask and how to ask them without leading respondents. For qualitative
data, interviewers need to elicit information from respondents while making them feel comfortable and must
create a trusting relationship with respondents while remaining neutral in attitude and appearance.
 The opportunity to physically test the tool with potential respondents.

18 August 2025 94
Training data collectors serves two purposes: building the skills of your data collection and
ensuring that your tool works as it should. You must always first test your tool to make sure
that:
 You will collect the data you intend to collect.
 Your questions are written using language that respondents and collectors understand.
 Your tools will not take too long to implement. You want to avoid situations where you put too great
a burden on respondents and/or risk them losing motivation and focus.
 Your tools appropriately explain to respondents the ethical norms and standards related to informed
consent, anonymity and confidentiality.
 Your data collectors have been sufficiently trained. They understand the tool’s instructions, the

18 August 2025 95
Revise and finalize your tools: incorporate all revisions into your final document.
Plan for implementation and data management: as you plan your data collection
activities, be sure to:
 Allow enough time for each data collection event.
 Choose a venue for interviews and focus group discussions that provides privacy and an
appropriate level of comfort.
 Identify how you intend to manage the data you collect. For example, consider who will be
responsible for entering data into the database(s) and conducing data quality checks, how you
will protect and store questionnaires and protect the privacy of respondents.
Step 3
Step 4

18 August 2025 96
 Data management is the process of managing data through the phases of its life.
Complete data management includes four primary components: entry, cleaning, storage
and security, and retention and disposal.
 Date entry means putting the data you collected into a form you can use by entering it
into an electronic database. Using a data base improves your ability to:
 Your choice of data base and database software will depend on what type of data you are
processing.
Managing data
Access, manage, and
share data
Improve data
security and
protection
Integrate data
more
effectively
Manage data
quality
Facilitate
timely
decision-
making

18 August 2025 97
When entering data, follow these steps:
Data entry
Step 1 Create a data entry protocol: to reduce the risk of inconsistent data entry, create a
standard data entry guidance that includes information on the data entry process
and the timing of data entry.
If necessary, identify your requirements for those entering data: while most data
entry is now conducted electronically, there still may be a need to input data by
hand. Any data entry protocols you create should clearly indicate whether those
inputting data require previous experience or training.
 All data entry staff should be trained on the objectives of your data collection
efforts, the data collection methods, the database itself, and the protocol you set
up for data entry.
Step 2

18 August 2025 98
To confirm that your data are correct, complete, and of the highest quality, teams should invest
in data cleaning to ensure that they are accurate.
 Data cleaning is when you detect and remove errors and inconsistencies from data to
improve its quality.
Cleaning data
Data cleaning
• Detect and remove
errors from data
Conducting quality
checks
• Randomly select and
compare data for
check on data-entry.
Identifying outliers
• Check for unexpected
entries in data.
Removing duplicate
entries
• Confirm that each data
record has a unique
identification number, and
no information is repeated.

18 August 2025 99
Data must be secure and protected against unauthorized changes, copying, tampering,
unlawful destruction, accidental loss, improper disclosure, or unauthorized transfer.
 Data storage and security measures will vary depending on:
Data storage and security

Data retention and de-identification
18 August 2025 100
When destroying data, consider whether you have paper or
electronic records to inform how you destroy the material.
Data disposal is the method you use to destroy data and
records will depend on the applicable laws, the
organization’s policies and donor requirements, local
operating context, sensitivity of the data that require
disposal, and the volume of data that requires disposal.
When data is no longer needed, all records and backups should be disposed of or
adjusted so that it is impossible to identify the data respondents.

18 August 2025 101
Retaining the data

If you choose to retain the data following the end of activities, you can conduct a de-
identification process to maintain respondents’ anonymity. Anonymization and
pseudonymization are two techniques that you can use to de-identify data.
Pseudonymization:
Replace personally identifiable information fields
with a code that protects a respondent’s identity.
However, with the use of a data ‘key’, the
individual’s identity can be accessed.
Anonymization:
Strip the data of any identifiable information

18 August 2025 102
Kahoot!
(Chapter 6)

18 August 2025 103
4. Analyzing MEAL Data
Phase 1:
Designing
logic models
Phase 2:
Planning
MEAL
activities
Phase 3:
Collecting
MEAL data
Phase 4:
Analyzing
MEAL data
Phase 5: Using
MEAL data both
externally and
embed learning.

18 August 2025 104
 Explain how your MEAL planning documents guide data analysis, visualization and
interpretation
 Describe the purpose and processes of quantitative data analysis
 Describe the purpose and processes of qualitative data analysis
 Describe the purpose and processes of data visualization
 Explain how analysis leads to appropriate interpretation and the development of
conclusions and recommendations
Objectives

18 August 2025 105
Data analysis, visualization, and interpretation
Data analysis
is the process of bringing order
and structure to the collected
data. It turns individual pieces
of data into information you can
use. This is accomplished by
applying systematic methods to
understanding the data—
looking for trends, groupings or
other statistical relationships
between different types of data.
Data visualization
is the process of putting
data into a chart, graph or
other visual format that
helps inform analysis. Data
visualization also helps you
interpret and communicate
your results.

18 August 2025 106
Data analysis is guided by your MEAL plan. Reviewing the MEAL Plan will tell you what data
you will analyze, when and how you will analyze them, and how you will use your results.
Data analysis
Quantitative analysis
analyzed using quantitative, statistical
methods and computer packages such
as Microsoft Excel or SPSS. The results
of quantitative data analysis are
numerical and easily visualized using a
graph, chart or map.
Qualitative analysis
most often done by reading through
qualitative data in the form of data
transcripts, such as notes from focus
group discussions or interviews, to
identify themes that emerge from the
data. This process is called content
analysis or thematic analysis. It can
be aided by software, but is most often
done using paper, pens and sticky notes.

18 August 2025 107
There are two kinds of quantitative analysis: descriptive and inferential (interpretive)
Descriptive data analysis is the analysis of a data set that helps you describe, show or
summarize data in a meaningful way so that patterns might emerge.
Inferential data analysis enables you to use data from samples to make statistical
generalizations about the populations from which the data were drawn.
 The kind of quantitative data you have determines what statistical analysis you can
conduct. Understanding your data begins with understanding variables.
 A variable is any characteristic, number, or quantity that can be measured or counted.
Variables can be independent (they stand alone and are not changed by other
variables) or dependent (depend on other factors).
Quantitative data analysis

18 August 2025 108
Data is classified into four fundamental levels of measurement: nominal data, ordinal data,
interval data, and ratio data.
Level of measurement
Level Description Examples Use scenario
Nominal data Data collected in the form of
names (not numbers) and which
are organized by category.
Gender, ethnicity,
religion, place of birth,
etc.
Nominal data can be counted, but not much else can be done.
Information collected from nominal data is very useful, even
essential, as it enables basic descriptions of your project.
Ordinal data Data that have an order to them.
They can be ranked from lesser
to greater.
Scales measuring levels
of satisfaction or levels of
agreement
Strictly speaking, ordinal data can only be counted. However, a
consensus has not been reached among statisticians about whether
you can calculate an average for data collected using an ordinal
scale.
Interval data Data expressed in numbers and
that can be analyzed statistically.
Temperature, time Distances between data points on an interval scale are always the
same. (This is not always the case with ordinal scales.) That means
that interval data can be counted and you can undertake more
advanced statistical calculations for interval data sets.
Ratio data Data expressed in numbers, with
the added element of an
“absolute zero” value.
Height, weight This means that ratio data cannot be negative. Because ratio data
have an absolute zero, you can make statements such as “one
object is twice as long as another.”

18 August 2025 109
Raw data are not particularly useful in their original form. You need to analyze the raw data
before you can determine whether your program is meeting its targets, use it to make
decisions, or start to communicate with your stakeholders.
 The table below shows how raw data collected from four respondents to a Delta River
IDP Project questionnaire are organized in the project database. By looking at this data
you can see trends but cannot make specific statements about the findings.
Using raw data
Respondent /
questionnaire identifies
Q1 (Age) Q2 (Number in
household)
Q3 (Use of
water points)
Q4 (Daily frequency
of water point usage)
Q5 (Distance
walked to water
point)
Q6 (Diarrheal
incident in last 3
months?)
V1R1 27 1 Yes 2 50 No
V1R2 53 1 Yes 1 1000 N/A
V1R3 19 2 No 3 400 Yes
V1R4 21 4 Yes 5 200 Yes

18 August 2025 110
There are three categories of calculations that are used to analyze data using descriptive
statistics:
Analyzing quantitative data using descriptive
statistics
Measures of
variability
Determine the extent
to which data points in
the data set diverge
from the average, and
from each other
(range, standard
deviation).
Measures of central
tendency
Calculate the center
value of data sets
(mean, median,
mode).
Measures of
frequency
Display the number
of occurrences of a
particular value(s) in
a data set (frequency
tables, cross-
tabulation tables).

18 August 2025 111
A measure of frequency indicates how many times something occurred or how many
responses fit into a particular category. You can analyze frequency by using two tools:
 Frequency tables
 Cross-tabulation tables
The tool will depend on whether you are measuring the frequency or the response values
of a single group (frequency table) or multiple groups (cross-tabulation table).
Measures of frequency

18 August 2025 112
A visual representation of the frequency of values in your data set.
 The Delta River IDP project conducted a questionnaire that asked: “I can access the water I
require to meet my household consumption needs”. The below frequency table provides an
clear summary of the results.
Frequency table
Question: “I can access the water I require to meet
my household consumption needs”
Number of
responses
Percentage
Strongly disagree 6 10 percent
Disagree 10 16 percent
Neither agree nor disagree 7 12 percent
Agree 25 42 percent
Strongly Agree 12 20 percent
TOTAL 60 100 percent

18 August 2025 113
While frequency tables help you analyze the frequency of data values according to a single
categorical variable, a cross-tabulation table will help you analyze the frequency of
responses according to multiple variables.
 A cross-tabulation table visualizes values’ frequency in an entire data set, including subgroups.
 Returning to the Delta River IDP project, rather than ask respondents about their level of satisfaction,
this table can compare the responses from large vs. small households.
Cross-tabulation table
Question: “I can access the water I require to meet my household consumption needs.” Response total
Response (large
HHs)
Response (small
HHs)
Strongly disagree 6 (10%) 4 (16%) 2 (6%)
Disagree 10 (16%) 8 (32%) 2 (6%)
Neither agree nor disagree 7 (12%) 4 (16%) 3 (9%)
Agree 25 (42%) 7 (28%) 18 (51%)
Strongly agree 12 (20%) 2 (8%) 10 (28%)
TOTAL 60 (100%) 25 (42%) 35 (58%)

18 August 2025 114
Strongly agree
Agree
Neither agree nor disagree
Disagree
Strongly disagree
10
2
18
7
4
3
8
2
2
4
An accompanying bar graph allows you to compare the responses of the two
groups.

18 August 2025 115
One of the most common ways to analyze frequencies is to look at measure of central
tendency. They help identify a single value around which group data is arranged.
 Three tools measure central tendency:
Measures of central tendency
• The average of a data set, identified by adding up all the values and dividing by the
whole.
Mean
• The middle point of a data set, where half the values fall below it and half are above.
Median
• The most common occurring answer or value
Mode

18 August 2025 116
The UNITAS randomly selected 10 IDP HHs in each village and physically measured how far
they walked to collect water. The raw data from HHs in village 1 is below.
Case Study
Indicator: “By Year 3, 85 percent of IDP HHs are located no more than 500
HH (Village 1) Distance walked (m)
R1 100
R2 300
R3 600
R4 400
R5 300
R6 700
R7 2,000
R8 300
R9 800
R10 100
Let’s use this data to try out the three different measures of central
tendency.
The mean:
 To calculate the average, you add up all responses to the
question about distance walked and divide it by the number of
respondents
(100+300+600+400+300+700+2,000+300+800+100)/10 = 560m

18 August 2025 117
To calculate the median, write out all the values in numerical order: 100-100-300-300-300-
400-600-700-800-2200. Then, cross off the first and last numbers in the row until you get to
the middle.
100-100-300-300-300-400-600-700-800-2200
100-300-300-300-400-600-700-800
300-300-300-400-600-700
300-300-400-600
 Data sets that contain an even number of values, like this one, will not have a middle value.
In this situation, you then take the average of the two values (300+400)/2 = 350.
 The median is not used as frequently as the mean, but it helps to double check whether the
mean provides a fair representation of the data. If you find that there is a large difference
between the mean and the median, then it could be a sign that there are outliers that are
skewing the mean.
Median

18 August 2025 118
The mode states what is the most commonly occurring answer or value in the data set. To
calculate the mode, write out a frequency table and identify the most frequently occurring
response value.
100 m = 2 responses
300 m = 3 responses
400 m = 1 response
600 m = 1 response
700 m = 1 response
800 m = 1 response
2,000 m = 1 response
Mode = 300 m
Mode

18 August 2025 119
To understand which calculation best expresses the central tendency of this data set, answer
the following three questions:
What type of data do you have (nominal, ordinal, interval or ratio)?
Does your data set have outliers and/or is it skewed?
What are you trying to show from your data?
In analyzing the data set, respondent 7’s data point is an outlier. In a larger data set, this
would not have a large impact on the mean like it does in this project. The median is useful
when the mean does not fall in the middle of your data set. When measuring the central
tendency of a numerical data set that is skewed, either use the median or both the median
and mean together. Generally, using more than one measurement brings more clarity to your
analysis.
 Why not use the mode? The mode is not often used to analyze numerical data sets. The
mode is more helpful with nominal data sets.
Which measure of central tendency should you
use?

18 August 2025 120
Measures of variability are the third set of calculations used to analyze data using descriptive
statistics. They tell you the spread or the variation of the values in a data set.
 To calculate the data set’s variability, we use the range and the standard deviation.
Measures of variability
Range:
the difference between the lowest and the
highest values of a data set. This is calculated
by subtracting the lowest value in the data set
from the highest value. Returning to the case
study, the longest distance walked is 2,000m
and the shortest is 100m. Thus, the range is
1,900m (2,000 - 199 = 1,900).
Standard deviation:
calculates how far responses differ (deviate)
from the mean (average). A high standard of
deviation indicates that the data set’s values
differ greatly from the mean. A low deviation
means that values are close to the mean. A
zero deviation means that the values are
equal to the mean.

18 August 2025 121
Descriptive statistics may not be enough to satisfy your analysis needs as you will want to
know whether the patterns you see in your sample can be true for the wider population. You
may also want to know whether the project is causing the changes that you are seeing.
 Inferential statistics are only possible when you have a good random sample that
generates high-quality data.
An inferential analysis helps you:
Inferential analysis
Compare the significance of
differences between groups:
Determining whether the
differences that exist between
subgroups are large enough
to matter.
Examine the significance of
differences between variables
to determine correlation and,
potentially, causation:
Determining whether your
activities contributed to the
changes you are seeing.

18 August 2025 122
1. Exploring the significance of differences between subgroups: t-tests, analysis of variance
(ANOVA), and chi-square tests help you determine whether the differences between the descriptive
statistics for subgroups are significant. Some inferential statistics calculate whether differences in
frequencies are significant, while others calculate whether differences in averages are significant.
Statistical tests
Analysis method Description Example questions
T-test ● The t-test compares the average for one subgroup
against the average for another subgroup.
● It can also compare differences in averages at two
points in time for the same subgroup.
● If the result of the test is statistically significant, you can
potentially consider it as a project impact.
“Is the average distance walked to collect water at the end of the
project significantly different from the average distance walked at
the beginning of the project?”
Analysis of
variance
● The ANOVA test compares the average result of three or
more groups to determine the differences between them.
“Does the average distance walked to collect water vary
significantly between villages 1, 2, 3 and 4?”
Chi-square test ● The chi-square test works with frequencies or
percentages in the form of a cross-tabulation table.
● It helps you see the relationship (if any) between the
variables and to know whether your results are what you
expect to see.
You expect that the creation of new water points will improve
access to water and thus meet consumption needs for both large
and small households. A chi-square test helps you statistically
test this expectation by analyzing the information provided in the
cross-tabulation table in Figure 55.
“Is there a significant difference between the responses of small
and large households to questions about household
consumption needs?” “How significant is this difference?”

18 August 2025 123
2. Examining differences between variables to determine correlation and causation: The tests
described earlier can tell you whether there is a statistically significant relationship between two groups,
which may give you some early indication of the effects of your project. The limitation of t-tests, ANOVA
and chi-square tests are that they don’t tell you which variables influence that relationship and which do
not. This is where regression analysis can help.
 Regression analysis shows how changes to variable(s) affect other variable(s). Regression analysis
is a way of mathematically sorting out which independent variables impact your dependent variable. It
answers the questions: Which factors matter most? Which can we ignore? How do those factors
interact with each other? How certain are we about all these factors? For example, it could possibly tell
you the different correlations between reducing waterborne disease rates and prevention methods.
 It also gives you an understanding of correlation – a measure that describe the size and direction of
the relationship between two or more variables.
Statistical tests

18 August 2025 124
Correlation does not necessarily imply causation, which is when changes to one or more
variables are the result of changes in other variables. For example, if your analysis shows a
correlation between handwashing messaging, improved handwashing practices, and the
reduction of waterborne disease, you cannot necessarily say that your project caused these
changes. Causation is hard to prove.
 There are two strategies that can be used to increase your confidence that causation
exists between variables:
Causation
Counterfactuals and control groups:
These are most often used in impact
evaluations. The “counterfactual”
measures what happens to the “control
group,” a group of people who are not
involved or impacted by your project.
During analysis and interpretation, you
compare the results of your project
sample with the control group in an
effort to demonstrate causation.
Mixed-method approaches: increase
certainty if multiple methods of data
collection and analysis lead to the
same conclusion. Then you have
demonstrated stronger grounds for
causation. Mixed method approaches
are easier for projects with less
resources and capacity to design a
rigorous impact analysis.

18 August 2025 125
As causation is difficult to prove, contribution is less difficult and is used in situations where
rigorous sampling and data collection processes are not possible.
 Contribution analysis clearly outlines a contribution ‘story’ by transparently following these
six steps to demonstrate change:
Contribution: an alternative to causation
1. Clearly define
the questions
that need to be
answered
2. Clearly define
the project’s
theory of change
and associated
risks to it
3. Collect
existing
evidence
supporting the
theory of change
(your conceptual
frameworks)
4. Assemble and
assess your own
project’s
contribution
story
5. Seek out
additional
evidence where
necessary
6. Revise and
conclude the
contribution
story

18 August 2025 126
When considering your type of analysis and sampling decisions, keep in mind two common
errors, Type I and Type II.
 Type I error: Wrongly concluding that your project has had an effect on the target population when it
has not. This is also called a false positive. Type I errors are problematic when considering expanding
your project based on positive results.
 To avoid Type I errors, you will want to plan for a smaller margin of error and a higher confidence
level when you select your sample from which to collect data. However, setting your requirements
too high can lead to Type II errors.
 Type II error: This is the opposite of the Type I error. This occurs when you wrongly conclude that your
project has not had an effect on the target population when it actually has. This is also called an error
of exclusion or a false negative.
 To reduce the risk of Type II errors, you can increase your sample size. This may have
implications on your budget.
Quantitative analysis errors

18 August 2025 127
Qualitative analysis is working with words that combine to become ideas, opinions, and
impressions. The object of this analysis is to identify key themes and findings, including
among subgroups if you have them, from all the notes collected from interviews and focus
groups.
 Content analysis is a popular type of qualitative analysis which requires multiple reviews
of data (your content) to generate familiarity with your data to discern themes and
interpretations for your analysis.
 It begins with raw data that can range from recordings of interviews to notes from
focus groups. The raw data should be reorganized for easy review. Once this is
done, you need to complete the following steps on the next slide.
Qualitative data analysis

18 August 2025 128
Coding is a process that helps reduce the large quantity of qualitative data you have into
manageable units. This process is iterative, meaning that you will learn as you code. Reading
the data may trigger new ideas and will require you to read the data again.
There are two competing forms of coding, which are deductive and inductive coding.
Deductive develops codes before the data is reviewed while inductive develops codes while
reviewing. Deductive coding rarely identifies all of the codes you need to analyze your data.
Using a mixture of both types offers the most comprehensive results.
Step 1: Coding data
To begin coding, conduct a first
reading of your transcripts, including
note taking to help you later identify
themes.
On the second reading, you may be
comfortable with adding codes. A code is a
category label that identifies a particular event,
opinion, idea etc. Codes should be descriptive
that people understand their meaning but not
difficult to manage.
Once finalized, map codes into a
matrix to help visualize the data and
interpret their meanings.

18 August 2025 129
 As you read your data, you may need to match concepts and relevant quotations to the
codes you identified, which is called indexing.
 Indexing often aids in sorting through large amounts of qualitative data. When you index
your data, you essentially tag the content from your transcripts using the codes from the
previous step. Then, you create a list of those tags and where they are in the data in the
form of an index.
 This process allows you to:
Step 2: Index data
How often the
code appears
and where
Identify how
dense a code
is

18 August 2025 130
 After indexing, you begin to put the qualitative data into a form that ca be understood.
This is often a matrix, or a framework approach, which organizes your data according to
the categories that are useful to you.
Step 3: Frame data
Location Large HHs Small HHs
Village 1 Access: Generally ok, but need to visit water point often per day.
Consumption needs: No consensus on if 30 L per person per day is enough.
Some require more for washing and cooking.
Location: Still too far away for some. No consensus.
Quality: Smells and tastes different, but acceptable.
Access: Much better than before.
Consumption needs: Meets consumption needs 30 L per person per day.
Location: New location is not safe for children, so need to send adult or older child to
collect water. But happy overall with the fact that it is closer.
Quality: Smells and tastes different, but much better than before.
Village 2 Access: All agree that the new water point location is a great improvement.
Consumption needs: 30 L per person per day is insufficient
Location: Large families need more water on average and the new location
allows them to acquire water more easily.
Quality: No specific complaints.
Access: Some families believe some have more access than others in new location.
Consumption needs: 30 L per person per day meets needs
Location: Not as centrally located as it could be.
Quality: No specific complaints.

18 August 2025 131
 Data visualization is the process of showing your data in a graph, picture, or chart. It helps
share detailed insights into data in the quickest and most efficient way. This helps with:
 Consider the following steps to ensure that your products are effective, especially if you
intend to use data visualization to aid communication to stakeholders.
Data visualization
Analysis: Discovering relationships
between, and patterns in, the data.
Interpretation: Understanding and
reflecting on patterns in the data set
and then inferring new information
based on that interpretation.
Communication: Making technical,
statistical analysis understandable to
people with limited technical
knowledge, and sharing your
information in ways appropriate to
your stakeholders.
Build your visualizations
Design and test your visualization
with audience specific content.
Define the data visualization
content, such as identifying the
‘need to know’ content for each
stakeholder. Then, determine
where visuals would be the most
useful.
Define the stakeholder(s).
Step
1
Step
2
Step
3
Step
4

18 August 2025 132
Examples and use scenarios of data
visualization tools
• Shows multiple responses across different subgroups or points in time. Useful when presenting various responses for only a few subgroups. Not
appropriate when the responses given are numeric or equal 100% in total.
Bar chart
• Shows the variation in multiple variables or options across different subgroups on different questions or different points in time. Useful when comparing
parts of a whole across different subgroups. Not useful when totals do not equal 100% or when representing only one subgroup.
Stacked column chart
• Shows composition of data set when component parts add up to 100 percent. Useful when demonstrating the different subgroups or demographics
represented within a data set. Not appropriate with many data points or when total does not reach 100%.
Pie chart
• Shows the trends across different points in time. Useful when tracking change over many points in time. Not appropriate to show cumulative data or
when comparing multiple (more than five) different trends.
Line chart
• Shows the relationship between two continuous variables (i.e. amount of harvest or rating scale) or distribution within a data set. Useful when looking
for patterns or outliers and for correlation in large data sets. Not appropriate when using binary (yes/no, etc.) variables or with very few data points.
Scatter chart
• Shows the distribution of results across a geographic area, with greater distributions represented by greater (“hotter”) color intensity (in this case, red).
Useful when covering an entire region or district. Not appropriate to demonstrate change within a subgroup or between different points in time.
Heat map
• Shows the distribution with a range of numeric data. Useful when looking for the range to accompany an average value. Not appropriate when
presenting categorical data (data that can be divided into mutually exclusive groups) or multiple responses given or when tracking changes over time.
Line histogram
• Visually display a collection of key data points to monitor the status of a project. Can include multiple visualization tools as subcomponents.
Data dashboards

18 August 2025 134
There is no prescribed process for interpreting data, but there are several recommended
practices for improving your data interpretation. These include:
 Creating visualizations of your results to help people better understand and interpret your data, making
sure your visualizations are used to give the full picture of the data and are not misleading.
 Triangulating your data by presenting the results of both quantitative and qualitative analysis together.
 Convening a stakeholder meeting to interpret the data. This meeting should involve stakeholders with
different perspectives on the project. The incorporation of multiple perspectives in your interpretation is
critical to the creation of information that will be useful and trusted to help the project improve.
 Planning an adequate amount of time to analyze and interpret data.
 Ensuring roles and responsibilities around interpretation are clear.
Interpreting quantitative and qualitative data

18 August 2025 135
 While undertaking data interpretation, you need to consider your interpretation through the
same lens(es) you used to view data quality. This includes being mindful of validity,
reliability, and integrity.
 Validity: Your interpretation is considered more valid if you clearly demonstrate that it is
based on data that directly supports it.
 Reliability: Your interpretation will be considered more reliable if you can demonstrate the
consistency of your data analysis methods and their use across multiple data sets.
 Integrity: Your interpretation will be considered to have more integrity if you can
demonstrate that it is based on data collection and analysis processes that are relatively
free of error and bias.
Interpretative lens

18 August 2025 136
Your interpretation must consider that the type of data collected limits your ability to make
interpretations and reach conclusions.
 Limitations related to data type: with qualitative data, you must recognize and be clear that your data
represent only the perspectives of the people surveyed. They should not lead to broad generalizations.
Quantitative data is limited in breadth of information it collects.
 Limitations related to sampling: the types of sampling methods impact the analysis and interpretation.
Sampling may limit what generalizations you can derive.
 Limitations related to data quality: information you collect will never be perfect and may influence your
interpretation. Your interpretation must incorporate your understanding of any data quality issues.
 Limitations related to bias: bias takes on many forms, as explained in the next slide
Data limitations to consider during interpretation

18 August 2025 137
Bias
Data analysis bias when the researcher eliminates data that does not
support their conclusion or uses statistical tests that are not appropriate for
the data set.
Data interpretation bias when your interpretation does not reflect the reality
of the data, such as generalizing results outside of the sampled population,
making conclusions about causation when the sample and collection
designs do not make this possible, and ignoring Type I and II errors.
Sampling bias when certain types of respondents are more likely than others
to be included in your sample.
Data publication and communication bias occurs when those publishing or
reporting on project results neglect to consider all results equally and skew
the results.

18 August 2025 138
Kahoot!
(Chapter 8)

18 August 2025 139
5. Using MEAL Data
Phase 1:
Designing
logic models
Phase 2:
Planning
MEAL
activities
Phase 3:
Collecting
MEAL data
Phase 4:
Analyzing
MEAL data
Phase 5: Using
MEAL data
both externally
and embed
learning.

18 August 2025 140
 Identify the key principles of adaptive management, including how they are incorporated
into the MEAL cycle
 Describe how data are used in reporting and communication with stakeholders
Objectives

18 August 2025 141
 Adaptive management encourages and supports MEAL information being used as part of
ongoing project decision-making.
 Effective adaptive management collects and analyzes project monitoring and feedback
data to help project staff make collaborative, timely, and informed decisions to ensure that
project activities deliver intended impact to participants.
 Project managers need accurate, relevant, and timely information to
 Adaptive management also contributes to internal and external learning and holds
individuals and teams more accountable to stakeholders.
Adaptive management
And address
feedback
raised by
community
members
understand the
perspectives of
participants
and their levels
of satisfaction
with the project
inform ongoing
problem-
solving and
good
management
decisions
assess project
management

18 August 2025 142
Is your project designed to promote adaptive
management?
Does your project
have resources to
support learning?
Your project does this by…
● Providing the budget, resources and time for learning-related activities.
● Recruiting staff who show passion and curiosity, and who are willing to question the standard operating
procedures and take risks.
Are project
decisions
informed by
evidence-based
data?
Your project does this by...
● Promoting a safe environment for speaking up, even when opinions differ from the majority or team lead.
● Intentionally and appropriately using evidence from multiple sources in analysis and interpretation.
● Generating timely and accurate data to inform project design, planning and implementation.
● Using feedback from stakeholders as part of decision-making.
Does your project
accept and
encourage
change?
Your project does this by…
● Encouraging flexibility, adaptability and entrepreneurship.
● Revisiting logic models and implementation plans to promote learning.
● Promoting and rewarding innovation.

18 August 2025 143
To create a report that resonates with stakeholders and are useful:
Progress reporting
Consult your project
communications plan
and data flow map
Identify or develop
report templates
Identify donor
reporting
requirements

18 August 2025 144
Outcome Mapping

18 August 2025 145
 Everything we do is nested in a system
 The systems we work with are complex
 We can contribute to change in the human bits
Complexity

18 August 2025 146
 In a remote Himalayan valley, many babies and women die or are damaged in
childbirth.
 Despite government schemes for women to use hospitals, they deliver at home
because of distance to hospital, cost, few cars, geography, cultural reasons,
shame with male doctors, language, tradition, little formal education....
 Though untrained (in western terms), villagers trust and understand their
traditional midwife (dai) who is called to every delivery.
 That’s how it’s always been.
Case Story

18 August 2025 147
 NGO trying to improve maternal health in this valley
 Arranges government doctor to train local Dai
 Pre-test: very poor medical knowledge
 Two days of excellent teaching.
 A test of knowledge transferred.
 NO Increase in knowledge.
 Disappointed doctor has lunch with the DAIs offers
 Phone help. Goes away sadly.
 No output = Failure
An unsuccessful intervention

18 August 2025 148
 At output level: Failure
 DAIs knew no more about
high risk pregnancy.
 At outcome level: Huge
success
 Doctor relating to Dai.
Professional advice flowing
from hospital to village.
 Women referred from village
to hospital.
 Relationships between
doctor, nurses, midwives,
village women, husbands,
NGO
Beneficial coherence!
WHY OUTCOMES?

18 August 2025 149
 What do we dream of?
 Who is crucial to that dream?
 Who is crucial and we can work with?
 Strategic work so they move to new behaviours
 All together, their new behaviours make our vision.
Outcome Mapping

18 August 2025 150
Overview of Outcome Mapping

18 August 2025 151
Conventional thinking

18 August 2025 152
 Complex
 Unpredictable
 Non-linear
 Two-way
 Beyond control
Reality is…

18 August 2025 153
 A participatory method for planning, monitoring and evaluation
 Focused on changes in behaviour of those with whom the project or program
works
 Oriented towards social & organisational learning to support systemic change
Definition of Outcome Mapping

18 August 2025 154
3 principles inherent to Outcome Mapping

18 August 2025 155
 Sphere of influence
 Setting actor-centred boundaries
 Outcomes understood as changes in behaviour
 Contributions to outcomes, not attribution
Four core concepts in Outcome Mapping

18 August 2025 156
1. There is a limit to our influence

18 August 2025 157
1. There is a limit to our influence

18 August 2025 158
2. Defining Boundary Partners

18 August 2025 159
Example: Forect justice in Tanzania

18 August 2025 160
3. Outcomes as Changes in Behaviour

18 August 2025 161
 Recognise yourself as part of the system you are seeking to influence
 One among many factors affecting change
 You may have high or moderate level of contribution
 May not know until the evaluation
4. Contribution, Not Attribution

STEP 1: Vision

The large-scale development-related changes that the program hopes
to encourage

The broad human, social and environmental betterment in which the
programme is engaged

The ideal to which the program wants to contribute
• Guides the intervention
• Motivates and inspires the team
• Accountability-free zone
• The vision should remain relevant over a longer time.
• Defines the (ideal) ‘system’ in which you intend to support change
18 August 2025 163

18 August 2025 165
Example of a Vision Statement

Local authorities, communities, and international organisations in developing countries in
Africa recognize the value of HIV/AIDS intervention as an integral part of social & economic

development. Municipal, regional, and national governments actively support HIV/AIDS

prevention activities by formulating and implementing effective public health policies. Using

research findings, they have developed a comprehensive public health strategy to slow down

the infection rate. Formerly marginalized groups (e.g. women and youth) are organized into

advocacy groups that can effectively formulate their needs to policy makers. All groups have

access to reliable and relevant technical information about HIV/AIDS prevention and are able
to make informed choices. In essence, there are healthier, happier, and wealthier
communities.

18 August 2025 166
STEP 2: Mission

The mission is that “bite” of the vision on which the program will focus.

What can we bring to the system?

Essentially a sum of strategies

18 August 2025 167
In support of this vision and on behalf of its donors, the program will work in the areas of
research, dissemination, capacity building, & coordination. It will contribute to the production,
synthesis, & dissemination of research data, position papers, & other information that will
sensitize local & international actors to HIV/AIDS prevention. The program will seek to expand
the range of disciplines involved in HIV/AIDS research. It will enhance HIV/AIDS research
capacity in order to produce credible information for local, national, & international policy-
making & program development. It will promote an interest in HIV/AIDS research among new
researchers by providing research fellowships, mentorship, & training opportunities. The
program will contribute to the development of linkages between Northern & Southern
researchers & encourage partnerships between research organizations, advocates, & decision
makers. It will increase its visibility & credibility among the donor community & will convince
them of the utility of supporting HIV/AIDS prevention.
Example of a Mission Statement

18 August 2025 168
• About the future
• Observable
• Idealistic
• Not about the Program
• Feasible
• Identifies activities and relationships
• About the Program

18 August 2025 169
Describe the vision of your initiative
 Write or draw what the ideal change is you want to see in the
world.
 Who would be doing what (actions, interactions and relationships)
differently?
 This is your chance to think big and dream big!
 The vision is outside of your ‘accountability’, but is there to prompt
creativity and inspiration. It helps us to set the scene, and to invite
multiple perspectives, needs and intent.
 You can write full sentences, create a set of bullet points, or do
drawings.
Practical Exercise - 1

18 August 2025 170
 Create your mission. What is the unique contribution, niche, strategies and capacities
that your initiative will bring to the vision? Who will you collaborate with?
 You can jot down a few notes, write full sentences, create icons to represent strategies.

18 August 2025 171
 Do a quick brainstorm of all of the individuals, groups or organisations that play a
role in, are affected by, could be affected by, or that may affect the initiative you
have selected.
 Guiding questions:
1. Who can the project influence or support to stimulate change?
2. Who can the project work with directly?
3. Whose actions or interactions are important for achieving sustained change?
4. Who has information, knowledge, resource or influence that can contribute to
change?
Before developing your vision and mission, you
need to identify they key actors

18 August 2025 172
STEP 3: Boundary partners
The individuals, groups & organisations with whom a programme interacts
directly to effect change and can anticipate opportunities for influence.

18 August 2025 173
STEP 3: Boundary partners
Stakeholders Vs. Boundary Partners
Stakeholder Boundary Partner
Interest in, affected by or whose
action affect the intervention
outcomes
Crucial to contribute to the vision
Many of them Want to see them change
Don’t/can’t work with them all Have an opportunity to work with them or
otherwise influence them
Few enough to monitor
Can be individuals, organisations, groups
(or categories)

18 August 2025 174
STEP 4: Defining Outcome Challenge

18 August 2025 175
 How the behaviour, relationships, activities, or actions of the Boundary Partners would change if
they reached their full potential as facilitators of change
 What are ideal actions, behaviours, relationships of Boundary Partners?
 ONE outcome challenge per Boundary Partner
 The challenge is for the program to help bring about these qualitative changes
 Answers to questions such as:
1. In order to contribute to the vision, how will the Boundary Partner be behaving or acting
differently?
2. What new relationships will have been formed?
3. How will existing ones change?
Outcome Challenge

18 August 2025 176
Example: Forect justice in Tanzania

18 August 2025 177
Village governments work with Village Assemblies, especially poorer members and women, to
develop village land use plans and village land forest reserve management plans. Plans are
implemented and regular updates are provided on the implementation of land use and forest
management plans and issues that arise are discussed during Village Assembly meetings. Data on
expenditure and revenue on forest management are shared and decisions are made on how to
disburse funds at the Village Assembly meetings and information is publicly available. Resources
are allocated for forest management activities. They have good relations with other authorities such
as the District Forest Office and the Forest Surveillance. Units and are confident that they would
receive support from relevant law enforcement agencies to address illegal forestry activities should
the need arise.
Example of Outcome Challenge

18 August 2025 178
STEP 5: Progress Markers
What are the things you would see along the way before you get to the Outcome
Challenge?
Close to the dream
BP choose to behave in different
ways proactively – Bulk of change
you would see in life of project
BP doing in response to project. E.g.
Coming to events, sharing interests
…

18 August 2025 179
 Describe a change pathway (progression of change).
 Describe nuanced change.
 Help us answer the question: “How can we know that we are moving towards
the ultimate, broader outcomes?”
Progress Markers

18 August 2025 181
 Step 1: Select a boundary partner
 Step 2: Describe the outcome challenge
 Create a paragraph of actions / interactions for the outcome challenge for the selected
boundary partner by answering the question: What are 3 or 4 important things you would
see the boundary partner doing if they were optimally contributing to the vision?
 To generate ideas, ask yourself:
1. Ideally, in order to be making its optimum contribution to the vision, how would the
boundary partner behave? What would the boundary partner be doing? With whom?
What interrelationships, activities and patterns of behavior is this actor engaged in and
committed to? What is this actor producing? With whom does the actor cooperate?
2. Are all the outcome challenge items expressed as observable actions?
3. Do they describe transformed patterns of behavior and relationships of the selected
boundary partner? Are they sufficiently ambitious? Do they represent a real challenge?

18 August 2025 182
 Imagine the actions and interactions that the boundary partner might engage in to move towards the outcome
challenge. What does that change look like? What would the boundary partner be doing differently along the
way? What concrete, tangible actions and interactions (relationships) would they be taking? Brainstorm your
ideas and then write them down in the progress marker table below. Remember to order those changes from
those that are more immediate, to ones that are more complex and represent deeper, transformational change.
 Some sub-questions to help you think of change:
1. What can you see happening as the actor moves toward playing a role that contributes to the outcome
challenge?
2. What does the boundary partner do better or differently?
3. What relationships does the boundary partner become engaged in and with whom?
4. What actions does the boundary partner take that suggest learning, adaptive management, commitment and
ownership are becoming established in the boundary partner’s way of operating?

18 August 2025 183
STEP 6: Strategy Maps (one per BP)

18 August 2025 184
 How does your team or organization stay relevant, viable and effective?
 8 practices:
1 - Prospecting for new ideas, opportunities, and resources
2- Seeking feedback from key informants
3- Seeking sustained support for your work
4- Assessing and (re)designing products, services, systems, and procedures
5- Checking in with Boundary Partners to add value
6- Sharing your experiences with the world
7- Experimenting to remain innovative
8- Engaging in organisational reflection
STEP 7: Organisational Practies

18 August 2025 185
 Systematic collection of data on outcomes and performance
 Engage in a regular learning & improvement cycles
 Understand (and celebrate) progress along the way and your own contribution
to change
 Encourages the program to challenge itself (critical reflection)
3 areas of Monitoring:
1. Outcome Journal: The progress of external partners towards the
achievement of outcomes (Progress Markers)
2. Strategy Journal: The internal performance of the program (Strategy Maps)
3. Performance Journal: The program's functioning as an organizational unit
(Organisational Practices)
Outcome Mapping Monitoring

18 August 2025 186
 What should we keep doing?
 What do we need to change in order to have more leverage?
 Are we still working with the right BPs?
 What strategies/practices do we need to add?
 What strategies do we need to end?
 What should be evaluated in more depth?
Critical questions?
(For example for yearly review)

18 August 2025 190
OMLC Online 2021: Using Outcome Mapping in the SAHA Program in M
adagascar - YouTube

18 August 2025 191
OUTCOME HARVESTING

USEFUL WHEN: NOT USEFUL WHEN:
• Focusing on finding outcomes / impacts
• Learning from your M&E activities
• Programming context is unpredictable
• Purpose of the project / activity is social
change
• Measuring increased knowledge
• Measuring direct service delivery,
e.g. how many times you did an
activity

6 Steps of Outcome Harvesting
1. Design the
Harvest
2. Review
Documentatio
n
3.Harvest
4.Evidence
5. Analysis &
Interpretation
6. Use the
information

18 August 2025 194
 A tool to identify, formulate, substantiate, analyse and interpret outcomes to
answer useful M&E questions.
 A tool to monitor if a programme or initiative has contributed to outcomes?
 Especially useful in complex settings
How is it different from Outcome Mapping?
1- During data collection, OH does not start from the intentional design/planning
2- Harvesting is an open process to identify the changes
3- Then you work backwards to identify the contribution
Outcome Harvesting

SOCIAL
ACTOR
Individual
Organisation
Institution
Behaviour Relationships
Actions,
activities
Policies,
laws and
practices
Group or
community
CHANGING
Outcome defined as a change in a social actor

18 August 2025 197
Main Players in Outcome Harvesting
Players Role
Users The individual(s) who require the findings of an Outcome
Harvest to take action. This may be one or more people
within the organisation or third parties such as a donor.
Change Agents Individual, project or organisation that influences a social actor
Social Actors Individual, group, community, organization, or institution that changed in
part because of the change agent.
Human Sources The person(s) who are most knowledgeable about an outcome and
motivated to describe them.
Harvester Person(s) responsible for facilitating the Outcome Harvest
process/coaching others in formulating outcomes

18 August 2025 198
 An observable and significant change in a social actor’s behaviour
 that has been achieved...
 ... and that has been influenced by your intervention.
How to formulate an Outcome Statement?

18 August 2025 199
An Outcome Statement has 3 elements

18 August 2025 200
Example of Outcome Statement
Specific, verifiable
facts
Includes opinion of
sources
Specific, verifiable
facts

18 August 2025 201
 A short narrative(1-3 sentences) describing the significant change in a social
actor.
 The outcome description starts with the date and then the actor who vchanged.
Outcome Statement

18 August 2025 202
 WHAT IS A SIGNIFICANT CHANGE? = A new practice, a break-through from
old habits, a step towards a long-term change. The significance comes from
the context of the programme.
 Explain why the outcome is important and add context information.
 Include information related to the situation prior to the change.
 Compare/relate the achieved change to the expected goals of the programme
Significance Description

18 August 2025 203
 Short description of the activities and outputs of your programme that plausibly
contributed to the change in the social actor.
 The contribution can be partially, indirectly and even unintentionally.
 Provide quantitative information: e.g., “4 meetings were held, “6 researches
conducted” etc.
Contribution

18 August 2025 204
Are not about what you did!
Outcomes
They are about who changed because of
what you did!
NOT just big, intended outcomes. ALSO unintended and minor outcomes
NOT just positive ones, ALSO negative ones
NO value statements, ONLY observable facts
Make it understandable for an outsider who doesn’t know your organisation or topic

Don’t Do
Use active verbs
Greater awareness…
Empowered women…
Community ownership…
Reduced conflict…
Increased collaboration…
Governmental
commitment…
Gender sensitivity…
Equal access…
Budgetary transparency…
Active participation…
Poverty alleviation…
Strengthened capacity…
Signed an agreement
Invited to a meeting
Participated in
Used
Worked together
Promoted
Published

1
Design
Based on users and
users, agree on:
-Outcome Harvesting
questions
-Process: what
information, from
whom, how, when
and with what
resources
2
Document
Review
Identify and
formulate draft,
potential
outcome
statements from
documentation
3
Engage human
resources
Review the
potential outcome
statements and
identify and
formulate new
ones
6
Support
the use of
findings
After the prime
questions are
answered so the
users make better
use of the process
and findings
6 Steps of Outcome Harvesting
Ensure accuracy or
deepen
understanding or
both so that the
whole set of
outcome statements
are credible enough
for the intended uses
4
Substantiate
with external
sources
Organising outcome
statements so they are
manageable and then
providing evidence-
based answers to the
prime questions
5
Analyse and
interpret

18 August 2025 207
 To review or complete draft outcomes
 To harvest (more) outcomes
 Engage those who are most knowledgeable about the changes...and are
motivated to share what they know
 The harvester needs to go back and forth with the sources in order to develop
credible enough outcomes
Step 3: Engage Sources

Confirmation of the substance of an outcome description by an
informant knowledgeable about the outcome but independent of the
change agent.
Britt & Wilson-Grau, 2013
Step 4: Substantiation
(Documentation and validation)

Why substantiate?
Why provide documentation and evidence?
Why validate outcomes?

Monitoring Evaluation Accountability and Learning Powerpoint by Abraham

More Related Content

Similar to Monitoring Evaluation Accountability and Learning Powerpoint by Abraham (20)

More from Abraham Lebeza (20)

Recently uploaded (20)

Monitoring Evaluation Accountability and Learning Powerpoint by Abraham

Editor's Notes