Monitoring and impact assessment tools

Monitoring and impact assessment tools for extension
interventions
ICAR-National Dairy Research Institute
Karnal-132001, Haryana (India)
B.S.Meena
bmeena65@gmail.com

Monitoring and Impact ........... Extension Interventions 2

Particular Input output Impact
Community
Investment Route
(Donations to, and
Partnerships with community
organisations)
How, Why, What,
Where
Social output
Business output
Leverage
Social Impact
Business Impact

Reproductive
problems
Problem severity Scores
Indigenous cow CB cow Buffalo (Mu) Buffalo (ND) Total
Anestrus
Repeat breeding
Abortion
Still birth
Retention of placenta
Dystocia
Cervico-Vaginal
prolapse
Uterine prolapse

Reproductive
problems
Problem severity Scores
1 2 3 4 5
Anestrus minor problem (<60
days)
60 days 90 days 120 days ≥ 150 days
Repeat breeding <3 services ≥ 3 services 4 services 5 services more than 5
services
Abortion 2 – 5 % 6 – 10 % 11 – 15 % 16 – 20% >20 %
Still birth No problem Up to 2.5% 5. -5 % 6 – 10% >10%
Retention of placenta minor problem up to
12 hours
12-24 hours 24-48 hours 48-72 hours More than 72
hours
Dystocia No assistant Slight assistant Needed assistant Considerable force Extremely
difficulty
Cervico-Vaginal
prolapse
Minor prolapse Intermittent
prolapse of vagina
especially when
recumbent
Continuous
prolapse when
recumbent and
stand
Continuous
prolapse of vagina
along with bladder
and cervix
Continuous
prolapse with
tissue damage by
trauma, infection
or necrosis
Uterine prolapse Minor prolapse Prolapse of Uterus Prolapse of both
uterine horn &
treatment within 24
hrs
Prolapse of both
horns and tissue
damage, infection

INTRODUCTION
The Global Consultation on Agricultural Extension observed that
monitoring and evaluation are important yet frequently neglected
functions in most organizations (FAO, 1990). In the worldwide survey
of national extension systems, it was found that only about one half
of all national extension systems have some type of monitoring and
evaluation capacity.
National extension systems should be strongly encouraged to
establish and use monitoring procedures and evaluation studies both
to improve extension performance and to communicate the results of
extension programmes to policy makers and clientele being served.
Extension systems need to carefully consider monitoring,
management information needs, sources of information, and a
management information system

Monitoring
• Monitoring is the systematic process of collecting, analyzing and using
information to track a programme’s progress toward reaching its
objectives and to guide management decisions. Monitoring usually
focuses on processes, such as when and where activities occur, who
delivers them and how many people or entities they reach.
• Monitoring is conducted after a programme has begun and continues
throughout the programme implementation period. Monitoring is
sometimes referred to as process, performance or formative evaluation.
Monitoring is the ongoing measuring or gauging of progress or impact
of extension, it stands in clear contrast to the more customary
“summatory” evaluation, which is usually conducted at the end of a
program and of which the evaluation results are normally of interest
only for managers or managerial decisions.

Principles of monitoring
1. Monitoring must be simple.
2. Monitoring must be timely.
3. Monitoring must be relevant.
4. Information provided through monitoring should be dependable.
5. Monitoring efforts should be participatory.
6. Monitoring must be flexible.
7. Monitoring should be action oriented.
8. Monitoring must be cost-effective.
9. Monitoring efforts should be top management oriented.
10. Monitoring units represent specialised undertakings.

EVALUATION
• Evaluation is systematic and objective assessment of an ongoing or a completed
project. It has a wider scope than monitoring. It focuses on design, implementation and
outcomes of the project. The aim of evaluation is to determine the relevance and
fulfillment of the objectives and identifying the impact.
• Evaluation is a process of systematically assessing the operation and/or outcomes and
impacts of a program or project by collecting evidence to determine if certain
acceptable standards have been met and to answer other relevant questions. This
implies that clear, measurable objectives are created for each program or project prior
to its implementation. Evaluation results based on these predetermined objectives, as
well as assessments of unintended consequences, are used to improve the program or
project, or to decide that it should be disbanded. Evaluation is both an art and a
science.

Monitoring Evaluation
Clarifies program objectives Analyses why objectives were or were not
achieved
Links activities and resources to
objectives
Assesses contributions of activities in
achieving results
Translates objectives into performance
indicators and sets benchmarks
Examines implementation process and
outcomes
Routinely collects data on these indicators
and compares results with targets
Explores unintended results, if any
Reports progress to managers and alerts
them to problems
Provides lessons, highlights program
potential, and
offers recommendations for improvement
Complementarities between Monitoring and Evaluation

COMPARISON BETWEEN MONITIORING AND EVALUATION
Basis for
comparison
Monitoring Evaluation
Meaning Monitoring refers to a routine
process that examines the
activities and progress of the
project and also identifies
bottlenecks during the process.
Evaluation is a sporadic
activity that is used to draw
conclusion regarding the
relevance and effectiveness
of the project or program.
Related to Observation Judgement
Occurs at Operational level Business level
Process Short term Long term
Focuses on Improving efficiency Improving effectiveness
Conducted by Internal Party Internal or External Party

Monitoring and evaluation are important because
M&E results in better transparency and accountability
M&E helps organizations catch problems early
M&E helps ensure resources are used efficiently
M&E helps organizations learn from their mistakes
M&E improves decision-making
M&E helps organizations stay organized
M&E helps organizations replicate the best projects/programs
M&E encourages innovation
M&E encourages diversity of thought and opinions

Monitoring and evaluation in extension
•Monitoring helps to ensure that programs are implemented in accordance
with their design and objectives, and helps answer questions such as “Are
we doing the right thing? Are we doing it right?” Extension managers use
monitoring to track progress by gathering periodic information on project
inputs and activities and, based on data, adjust an ongoing program’s
personnel, resource allocation, and/or staff recognition, and often are
linked with formal impact assessments.
•Most extension systems have set up a data collection system on what
extension program is offered to whom, where and how many benefitted, etc.
Generally, extension managers track resources (e.g., funds, personnel, and
supplies) and processes (e.g., occurrence of meetings, demonstrations,
and publications).

Selecting appropriate evaluation tools
• It is important to select appropriate tools and techniques. As
with all decisions related to evaluation, the selection of tools
and techniques should be consistent with the purpose, user,
and intended use of the results.
Common Evaluation Tools and Techniques by Program
Stage

Types of
Studies
Typical Questions Answered Examples of
Evaluation Tools
and Techniques
Needs
Assessment
What are the felt and unfelt
needs of the audience?
Surveys
Focus Groups
Feasibility
Study
Can extension address these
needs?
Do they fit with extension’s
mission?
Observation
Content Analysis
(e.g., of office records)
Baseline
Study
Is the program or project
socially, economically,
environmentally feasible?
Economic Analysis
(e.g., benefit/cost
analysis)
Program Stage : Planning Stage

Types of
Studies
Typical Questions
Answered
Examples of Evaluation Tools
and Techniques
Formative
Evaluation
Is the program meeting its
objectives of intended
outcomes?
Annual Monitoring Reports
(e.g., staff time and activity
reports,
crop yield, seed cost)
Is the audience satisfied with
the program?
Adoption Patterns for New
Technology
Program
Monitoring
Are the media delivering
program messages?
Evaluative Studies of Knowledge,
Attitude, and Behavior Change
Customer Satisfaction Surveys
Content Analysis of News
Releases
Implementation Stage

Types of
Studies
Typical Questions
Answered
Examples of Evaluation Tools
and Techniques
Summative
Evaluation
Is the program cost
effective?
Surveys
(e.g., personal interviews,
telephone
surveys, mail surveys, online
surveys)
Impact
Assessment
Has the program addressed
the needs or gaps identified?
Pre- and Post-project Data
Analysis
Is the program achieving
desired outcomes?
Cohort Studies
Concluding or Results Stage

What is Impact Assessment?
• Impact Assessment is a means of measuring the
effectiveness of organizational activities and judging the
significance of changes brought about by those activities. It
is neither Art nor Science, but both. Impact assessment is
intimately linked to Mission and articulate impact is a
powerful means of communicating, internally and externally,
the contribution of activities to the Mission of an operation/
organization/ programme/ project etc.

Approaches of Impact Assessment
• 1. Ex ante impact assessment: Usually done to know what the
impact of an intervention might be. It is done at need analysis
and planning stage of project cycle. Here the future benefits
of the project/intervention will be calculated in terms of
economic benefits, productivity increase, effect on
environment etc.
• 2. Ex post impact assessment: Usually conducted to know to
what extent and how an intervention brings out a change it
was intended to. This is done after the technology/
intervention has been widely adopted by farmers/
beneficiaries. It is mostly used to know the impact of
technologies.

Difference between Monitoring, Evaluation and Impact Evaluation
Monitoring Evaluation Impact Evaluation
Measures on-going
activities
Measures performance of
project against objective
the Measures changes:
positive or negative,
intended or not
Important during
implementation of the
project
Important in the middle or
at the end of the project
Can be included at all
stages and/ or after the
end of the project
Focus on intervention Focus on intervention Focus on affected
population
Focus on outputs Focus on outcomes Focus on impact
‘What is being done’? What has happened? Did
we achieve what we
intended to achieve?
‘What has changed? For
whom? How significant it is
for them?’

Quantitative and qualitative tools for impact assessment:
Quantitative tools Qualitative tools
Structured interview
schedule
Key informant and semi-structured
interviews
Surveys Participatory rural appraisal
Questionnaires Rapid rural appraisal
Telephone Focus group discussions
Participant observation
Case study

Criteria to select statements and Indicators
Statements Should be simple ,clear, and direct towards objectives
Should be unambiguous
Should not be factual
Should avoid double negative items
Should not use word like often, always, never, sometime etc.
Avoid double-barreled items, which is one item asking two questions
Avoid statements that are irrelevant to the psychological object
under consideration

Steps to be followed for finalize statements
1. Select the statements from the books, literature,
articles, personal experience, experts etc.
2. Item relevancy analysis by the expert judgement
3. Normalization of each indicators
4. Assigning weight to each indictors and ask to
respondents

Method Convert qualitative data to quantitative
• Adoption of Improved kid management practices by the
Goat farmers
• Here, Researcher can use 3 code for Adopted continued and
2 code for Adopted discontinued and 1 code for never
adopted improved practices of kid management practices of
Goat. So by using these code we could analysis that how
many goat farmers are adopted , never adopted etc.

Sl.
no
Practices Adopted
continued
(3)
Adopted
discontinued
(2)
Never
adopted
(1)
1 Do you remove mucous membrane around mouth and nostril and clean
with cloth?
2 Do you cut the naval cord and apply antiseptic?
3 Do you feed colostrum as early as possible (<30 mint.) ?
4 Do you provide milk according to body weight?
5 Do you provide drug for deworming?
6 Do you Provide feed (green/concentrate) from 15 days onwards?
7 Do you Provide clean and fresh water?
8 Do you castrate of male kid before 2 months (for sale purpose)?
9 What method do you follow for castration of male kid?
10 Do you provide proper bedding material in winter?

Impact Assessment of Livestock Based Intervention of the Dairy Farmers
Sr No. List of Dairy Based Intervention Number Number of Beneficiary
1 Balanced Feeding
1 Bypass fat supplementation
3 Field Visit
4 Bio Security and Hoof Management
5 Dairy Mela
6 Animal health camp
7 Ectoparasite Control
8 Endoparasite Control(Deworming)
9 Mastitis Control Protocol
10 Targeted Infertility treatment
11 Calf Care
12 Fist aid/ Emergency Management Kit
13 Mineral Mixture
14 Dairy Education at Doorstep of Farmer
15 Kisan Sangoithi
16 Value addition of milk
17 Extension Literature Distributed
18 Training Programme organized

Economic Impact Assessment of Balanced feeding
Sr No. Particulars n=
(Population of Livestock Keeper)
1 Cost of Balanced feeding ( Month/Year) In Rupee
2 Average milk yield before Intervention In liters/day
3 Change in milk yield after providing Intervention In liters/day
4 Return from the Milk yield (In month, Year) In Rupee
5 Increase in reproductive efficiency In Rupee
6 Increase in Value of Animal In Rupee
7 Total Expenditure In Rupee
8 Gross Return In Rupee
9 Net Return = Total Expenditure - Gross Return In Rupee
10 B:C Ratio - Gross Return /Total Expenditure In Rupee

Economic Impact Assessment of Bypass fat supplementation
Sr No. Particulars n= (No. Livestock Keeper) and Unit
1 Cost of Bypass fat supplement (Month/Year) In Rupee
2 Average milk yield before intervention In kg
3 Change in milk yield after intervention In kg
4 Return from milk yield In kg
5 Increase in productive efficiency In Rupee
6 Increase in value of animal In Rupee
7 Cost on man power involved In Rupee
8 Total Expenditure In Rupee
9 Gross return In Rupee
10 Net Return = Total Expenditure - Gross Return In Rupee
11 B:C Ratio - Gross Return /Total Expenditure In Rupee

Monitoring and impact assessment tools

More Related Content

What's hot (20)

Similar to Monitoring and impact assessment tools (20)

Recently uploaded (20)

Monitoring and impact assessment tools