monitoring and evaluation

Monitoring & Evaluation
Dr. Tarek Abd-Elkader Sallam

Monitoring
• Collection of
routine data
that measure
progress
toward
achieving
program
objectives.

Monitoring
• It is used to
track
changes in
program
performance
over time.

Monitoring
• Its purpose is to
permit
stakeholders to
make informed
decisions
regarding the
effectiveness of
programs and the
efficient use of
resources.

Monitoring
Examples of program elements that can be monitored:
• Supply inventories
• Number of vaccine doses administered
monthly
• Quality of service
• Service coverage
• Patient outcomes (changes in behavior,
morbidity, etc.)

Monitoring
Monitoring
Counting clients
Tracking IUD distributed
Collecting Data on clients

Monitoring
• Is an ongoing, continuous process
• Requires the collection of data at multiple
points throughout the program cycle, including
at the beginning, to provide a baseline
• Can be used to determine if activities need
adjustment during the intervention to improve
desired outcomes

Evaluation
1. Measures how well a program have met
objectives.
2. The extent of changes in outcomes due to
the program.
3. Measuring the “impact,” is commonly
referred to as “impact evaluation.”

Evaluation
• Data collection
• at the start of a
program
(baseline)
• at the end,
• at repeated
intervals during
program
• A control group
• to measure
whether the
changes in
outcomes can
be attributed to
the program
• A well-planned
study design

“Monitoring” or “Evaluation.”
• • The National Council of Population and
Development wants to know if the programs
being carried out in Province A are reducing
unintended pregnancy among adolescents in
that province.

“Monitoring” or “Evaluation.”
• USAID wants to know how many sex workers
have been reached by your program this year.
• A country director is interested in finding out
if the post abortion care provided in public
clinics meets national standards of quality.

M & E importance
• • Make informed
decisions regarding
program operations
based on objective
evidence

M & E importance
• • Ensure the
most effective
and efficient
use of resources

M & E importance
• Objectively assess the extent to which the
program is having or has had the desired
impact, in what areas it is effective, and
where corrections need to be considered

M & E importance
• Meet organizational reporting and other
requirements, and convince donors that
their investments have been worthwhile or
that alternative approaches should be
considered

Questions that M&E can answer
1. Was the program implemented as planned?
2. Did the target population benefit from the
program, and at what cost?
3. Can improved health outcomes be attributed
to program efforts?
4. Which program activities were more
effective and which less effective?

WHEN SHOULD M&E TAKE PLACE?M&E
continuous process that
occurs throughout the
program.
planned at the design stage
time, money, and personnel
calculated and allocated
Monitoring should be
conducted at every stage
data collected, analyzed, and
used on a continuous basis
Evaluations
Conducted at the
end of programs
planned at the start baseline data
5-10% of a project budget
should be allocated for
M&E.

M&E PLANS
M&E plan
Program is
modified
New
information is
needed
Revised
whenever

M&E PLANS
Typically, these plans include:
1. • The underlying assumptions on which the
achievement of program goals depend
2. • The anticipated relationships between
activities, outputs, and outcomes
3. • Well-defined conceptual measures and
definitions, along with baseline values
4. • The monitoring schedule

M&E PLANS
Typically, these plans include:
5. • A list of data sources to be used
6. • Cost estimates for the M&E activities
7. • A list of the partnerships and
collaborations that will help achieve the
desired results
8. • A plan for the dissemination and utilization
of the information gained

WHY ARE M&E PLANS IMPORTANT?
• State how a program will measure its achievements and
therefore provide accountability
• Document consensus and provide transparency
• Guide the implementation of M&E activities in a
standardized and coordinated way
• Preserve institutional memory

M&E PLAN COMPONENTS
1. • The introduction
2. • The program description and framework
3. • A detailed description of the plan indicators
4. • The data collection plan
5. • A plan for monitoring
6. • A plan for evaluation
7. • A plan for the utilization of the information
gained
8. • A mechanism for updating the plan

M&E PLAN COMPONENTS
INTRODUCTION
• The introduction to the M&E plan should
include:
• The purpose of the program, the specific M&E
activities that are needed, and their
importance
• • A development history that provides
information about the motivations of the
internal and external stakeholders

M&E PLAN COMPONENTS:
DESCRIPTION AND FRAMEWORKS
1- A
problem
statement
situation that needs changing
The affected
Causes
magnitude,
Impact on society

• The
program goal
and
objectives:
The goal is a broad
statement about a desired
long-term outcome
Objectives are statements
of desired specific and
measurable program results

Descriptions
of the
specific
interventions
duration,
geographic scope
target population

The list of
resources
needed
financial,
human,
Infrastructure (office space,
equipment, and supplies)

The
conceptual
framework
which is a graphic depiction of
the factors thought to influence
the problem of interest and
how these factors relate to one
another

The logical
framework
or results
framework
that links the
goal and
objectives to the
interventions

SMART”
objectives
Specific:
Measurable:
Appropriate: related to the goal
Realistic:
Timely:

• Here is a sample objective. Do you think it is
SMART?
Increase contraceptive prevalence by 15% in
women 30-49 years

Increase contraceptive prevalence by 15% in
women 30-49 years
1. Specific: Yes, the intended outcome of the
program is specified
2. Measurable: Yes, contraceptive prevalence is
measurable.
3. Appropriate: Unknown, because no program’s
goal
4. Realistic: Unknown, because no resources
5. Timely: No

INDICATORS
Indicators are clues, signs, or markers that
measure one aspect of a program and show
how close a program is to its desired path and
outcomes.
The M&E plan should include descriptions of
the indicators.

INDICATORS
Examples of indicators:
• Number of health workers trained in IUD insertion in the
past 12 months
• Percentage of women of reproductive age who are using
a contraceptive method at a particularpoint in time
• The number of maternal deaths per 100,000 live births in
a specific period

DATA SOURCE AND DATA COLLECTION
• Data sources
• The data collection plan

DATA SOURCE AND DATA COLLECTION
• Examples of data sources are:
1. • Birth registers
2. • Sentinel/demographic
3. • Surveillance
4. • Censuses
5. • Focus groups
6. • Household surveys

MONITORING PLAN
• The monitoring plan describes:
1. Components that will be monitored
2. How this monitoring will be conducted
3. The indicators that will be used to measure
results

EVALUATION PLAN
The
evaluation
plan
research design
identify whether
changes in outcomes
can be attributed to
the program

INFORMATION DISSEMINATION and USE
Help ensure that findings from M&E efforts are not
wasted because they are not shared.
The various users of this information should be clearly
defined.
Dissemination channels can include written reports, press
releases and stories in mass media, and speaking events.

FRAMEWORKS
• A conceptual framework useful for identifying and illustrating
the factors and relationships that influence the outcome of a
program or intervention.

FRAMEWORKS
Results frameworks diagram the direct causal
relationships between the progressive results
of the key activities all the way up to the overall
objective and goal of the intervention.
This clarifies the points in an intervention at
which results can be monitored and evaluated.

FRAMEWORKS
• LOGIC MODELS
• Provides a streamlined, linear interpretation
of a project’s planned use of resources and its
desired ends.

INDICATORS
• Is a variable whose value changes from the
baseline level at the time the program began
to a new value after the program and its
activities have made their impact felt. At that
point, it, is calculated again.

INDICATORS
• measures the value of the change in
meaningful units that can be compared to
past and future units.

INDICATORS
• focuses on a single aspect of a program or
project.
• This aspect may be an input, an output, or an
overarching objective, but it should be
narrowly defined in a way that captures this
one aspect as precisely as possible.

Types of INDICATORS
• measure the specific resources
• amount of funds allocated to the health
sector annuallyinputs
• measure the immediate results
• Ex. number of staff trained
outputs
• measure whether the outcome changed
in the desired direction
• example, contraceptive prevalence rateoutcomes

CHARACTERISTICS OF INDICATORS
• Produce the same results when used repeatedly to
measure the same condition or event
• Measure only the condition or event it is intended to
measure
• Reflect changes in the state or condition over time
• Represent reasonable measurement costs
• Be defined in clear and unmistakable terms

CHARACTERISTICS OF INDICATORS
Indicator
values
interpretable
timely
precise,
valid,
reliable.
comparable

LINKING INDICATORS TO
FRAMEWORKS

LINKING INDICATORS TO
FRAMEWORKS
A. Provision of support
and supplies to
community-based
distributors
B. Expanding family
planning services to
additional clinics
C. Clinical training for
providers
D. The development of a
checklist to monitor the
quality of care
E. Management training
for supervisors

GUIDELINES FOR SELECTING
INDICATORS
• Select indicators requiring data that realistically can be collected with
the resources available.
• Select at least one or two indicators (ideally, from different data
sources) per key activity or result.
• Select at least one indicator for each core activity (e.g., training event,
social marketing message, etc.).
• Select no more than 10–15 indicators per area of significant program
focus.
• Use a mix of data collection sources whenever possible.

DATA SOURCES
routine
• Vital registration
records
• Clinic service statistics
• Demographic
surveillance
nonroutine
• Household surveys
• National censuses
• Facility surveys

DATA COLLECTION
• The plan should include information for each
data source such as:
1. • The timing and frequency of collection
2. • The person/agency responsible for the
collection
3. • The information needed for the indicators
4. • Any additional information that will be
obtained from the source

DATA COLLECTION
• Some types of errors or biases common in data
collection are:
1. • Sampling bias: Occurs when the sample is not
representative
2. • Nonsampling error: All other kinds of
mismeasurement, such as courtesy bias,
incomplete records, incorrect questionnaires,
interviewer errors, or nonresponse rates
3. • Subjective measurement: Occurs when the
data are influenced by the measurer

DATA COLLECTION
data quality issues to consider:
Coverage:
Completeness:
Accuracy:
Frequency:
schedule:
Accessibility:
Power:

monitoring and evaluation

More Related Content

What's hot (20)

Similar to monitoring and evaluation (20)

More from Tarek Sallam (12)

Recently uploaded (20)

monitoring and evaluation