Monitoring_and_Evaluation_2.pdf

NATIONAL AIDS COUNCILS
MONITORING AND EVALUATION
OPERATIONS MANUAL

© Joint United Nations Programme on HIV/AIDS
(UNAIDS) 2002.
All rights reserved. This document may be freely
reviewed, quoted, reproduced or translated, in part
or in full, provided the source is acknowledged.
The document may not be sold or used in
conjunction with commercial purposes without prior
written approval from UNAIDS (contact: UNAIDS
Information Centre).
The views expressed in documents by named
authors are solely the responsibility of those
authors.
The designations employed and the presentation of
the material in this work do not imply the expression
of any opinion whatsoever on the part of UNAIDS
concerning the legal status of any country, territory,
city or area or of its authorities, or concerning the
delimitation of its frontiers and boundaries.
The mention of specific companies or of certain
manufacturers products does not imply that they
are endorsed or recommended by UNAIDS in
preference to others of a similar nature that are not
mentioned. Errors and omissions excepted, the
names of proprietary products are distinguished
by initial capital letters.
UNAIDS - 20 avenue Appia - 1211 Geneva 27 - Switzerland
Tel. (+41) 22 791 36 66 - Fax (+41) 22 791 41 87
E-mail: unaids@unaids.org - Internet: http:/www.unaids.org
UNAIDS/02.47E (English original, August 2002)
ISBN: 92-9173-227-3

NATIONAL AIDS COUNCILS
MONITORING AND EVALUATION OPERATIONS MANUAL

ii
Acknowledgements
We wish to express our gratitude to the numerous National AIDS Councils’ members, the
organizations and the individuals that contributed to this manual. The manual was prepared by
Professor David Wilson, PhD, and reviewed by practitioners and donors. The manual was also
officially reviewed by the International Partnership against AIDS in Africa (IPAA) in Dakar in
October 2001 and the UNAIDS Monitoring & Evaluation Reference Group (MERG) in
Geneva in November 2001. A final draft was reviewed by a UNAIDS, World Bank and Centers
for Disease Control and Prevention (CDC) steering group in Geneva in March 2002.
Acknowledgements

iii
Acronyms
BSS Behavioural surveillance survey
CBO Community-based organization
CDC Centers for Disease Control and Prevention
CPA Country Programme Adviser
DHS Demographic and Health Survey
FHI Family Health International
IPAA International Partnership against AIDS in Africa
M&E Monitoring and evaluation
MAP World Bank Multi-Country AIDS Program
MEASURE Monitoring and Evaluation to Assess and Use Results
MERG UNAIDS Monitoring and Evaluation Reference Group
MICS UNICEF Multiple Indicator Cluster Survey
NAC National AIDS Council
NAP National AIDS Programme
NGO Nongovernmental organization
OVC Orphans and other vulnerable children
PMTCT Prevention of mother-to-child transmission
PLWHA People living with HIV/AIDS
STI Sexually transmitted infection
UNAIDS Joint United Nations Programme on HIV/AIDS
VCT Voluntary counselling and testing
WHO World Health Organization
Acronyms

Monitoring_and_Evaluation_2.pdf

TABLE OF CONTENTS
A. INTRODUCTION 1
Who is this manual intended for? 2
What does this manual attempt to do? 2
What does this manual not attempt to do? 2
What is M&E? 3
What key M&E lessons have been learned? 3
What framework is suggested for M&E? 5
What M&E components exist and how well developed are they? 6
Who should do what? 7
NAC indicators and data sources 7
B. OPERATIONS PROCEDURES 9
Overview 9
Clarifying NAC's role and increasing its capacity to coordinate 9
Contracting an M&E consultant 9
Recruiting specialized M&E units and agencies 10
Developing an M&E manual with participatory approaches 10
Implementing the agreed M&E system 13
APPENDICES 1 TO 9: TOOLS 17
1. Illustrative indicators 18
2. Summary terms of reference for NAC M&E staff 20
3. Detailed terms of reference for NAC M&E consultant 21
3.1. Draft advertisement for NAC M&E consultant 24
3.2. Proposed NAC consultant selection scoring criteria 25
4. Summary terms of reference for specialized programme 26
activity monitoring entity
5. Planning, monitoring and evaluation form 27
5.1. Service delivery sub-section 29
6. Programme areas requiring quality-assurance checklists 30
7. Illustrative quality-assurance checklists for 31
interpersonal communication
8. Checklist for project managers in NACs and donor 32
agencies
9. Indicative budget for design of overall M&E system 33
and programme activity monitoring
APPENDICES 10 TO 12: FURTHER INFORMATION 35
10. Key sources for further information on M&E 36
11. Illustrative time frame for participatory planning process 37
12. Illustrative budget for participatory planning process 38
Table of contents v

1
A. INTRODUCTION
1. HIV/AIDS is the leading cause of
death in sub-Saharan Africa. More than 18
million Africans have died, more than 12
million African children have been
orphaned because of AIDS, and another 28
million Africans are living with the virus
today, the vast majority of them in the
prime of their lives as workers and parents.
Life expectancy is dropping, family
incomes are being decimated, and agricul-
tural and industrial efficiency is declining
because of the epidemic. African nations
and the international community have rec-
ognized how disastrous the epidemic is to
the African continent, and have concluded
that past efforts to wage war against the
virus have failed because: (i) there was
insufficient commitment and leadership to
fight the epidemic among nations both
inside and outside the continent; (ii) the
war was being waged with too few human
and financial resources; (iii) those pro-
grammes that were effective, often under-
taken by civil society organizations, were
rarely scaled up; (iv) resources were not
reaching communities; and (v) pro-
grammes were too narrowly focused on the
health sector. A new strategy has been
developed by African countries and the
donor community to wage war more effec-
tively. It is based on:
• defining national HIV/AIDS prevention,
care, treatment and mitigation strategies
and implementation plans through a par-
ticipatory process;
• establishing National AIDS Councils
(NACs) at the highest level of govern-
ment, with broad stakeholder representa-
tion from the public and private sector
and civil society;
• empowering stakeholders from the vil-
lage to national level with money and
The new approach relies on immediate
M&E of programmes to determine which
activities are efficient and effective and
should be expanded further, and which are
not and should be stopped or would benefit
from capacity-building.
decision-making authority within a
multisectoral framework; and
• using exceptional implementation
arrangements such as channelling
money directly to communities and
civil society organizations and con-
tracting out many administrative func-
tions.
2. The new approach emphasizes speed
(due to the nature of the epidemic), scaling
up existing programmes and capacity-
building, 'learning by doing' and continu-
ous project rework, rather than exhaustive
up-front technical analysis of individual
projects. The new approach relies on
immediate monitoring and evaluation
(M&E) of programmes to determine which
activities are efficient and effective and
should be expanded further, and which are
not and should be stopped or would benefit
from capacity-building.
3. This new approach is being supported
by a number of donors, including bilateral
agencies, the Global Fund to Fight AIDS,
Tuberculosis and Malaria1 and the World
Bank, which is committing US$1 billion
through the Multi-Country HIV/AIDS
Programs (MAP) for Africa2. In an era
when those inside Africa as well as around
the world demand performance and trans-
parency in the use of funding, programme
M&E is essential in order to:
• establish performance incentives for
programme implementers in both the
public sector and civil society;
Introduction
1 www.globalfundatm.org
2 www.worldbank.org/afr/aids/map.htm

2
• detect and address problems so that
project redesign and improvement
become standard operating procedures;
• provide early evidence of programme
effectiveness; and
• communicate to those infected and
affected by HIV/AIDS, in transparent
and objective ways, the effort being
made to improve prevention, care, treat-
ment and mitigation programmes.
4. M&E thus becomes a core part of the
fiduciary architecture of financial manage-
ment, disbursement and procurement,
which is the basis for the performance con-
tract on which the war against HIV/AIDS
is being waged. The design of programme
M&E has to change with this new
approach. M&E must be relevant, objec-
tive, transparent and, most importantly,
available as: (i) a source of information on
performance for the public and for donors;
and (ii) a management tool for implemen-
tation agencies in the public and private
sector, in civil society and for country
coordination mechanisms such as NACs.
M&E systems must also be fully funded
for at least 5-10 years, including local costs
and incremental operating costs, since lack
of sustained funding is a major reason why
monitoring and evaluation fail.
5. This manual is designed as a practical
toolkit and road map for practitioners to
use in designing and implementing pro-
gramme M&E. While published jointly by
UNAIDS and the World Bank, it has bene-
fited from extensive consultation and pilot-
ing among African countries and other
stakeholders who recognize the importance
of M&E as an essential weapon in the war
against HIV/AIDS.
Who is this manual intended for?
6. This manual is intended for:
• NACs, particularly those that are tak-
ing on the role of grant-provider, and
their public sector and civil society
implementing partners in sub-Saharan
Africa; and
• donor institutions that are involved in
the preparation, implementation and
M&E of HIV/AIDS programmes in
partnership with NACs.
What does this manual attempt
to do?
7. This manual attempts to:
• introduce key concepts;
• present simple, clear procedures, with
a checklist of the process, timing and
costs of building participatory pro-
gramme M&E for NACs;
• offer key tools that implementing part-
ners need for M&E; and
• provide examples of terms of reference
and other M&E management and
administration materials.
8. This manual emphasizes:
• the development of the overall M&E
system, in relation to the National
Strategic Plan; and
• programme M&E or the monitoring of
services provided through NACs and
their implementing partners.
9. It focuses on these components
because they are essential in order for
NACs to be accountable and they are the
least developed components of M&E sys-
tems.
What does this manual not
attempt to do?
10. This manual does not attempt to cover
all aspects of M&E. In particular, it does
not attempt to deal with topics that are well
This manual is designed as a practical
toolkit and road map for practitioners to
use in designing and implementing pro-
gramme M&E.
Introduction

3
• Condom
availability
• Trained staff
• Quality of services
(e.g., STI,VCT, care)
• Knowledge of HIV
transmission
Short-term and inter-
mediate effects:
• Behavior change
• Attitude change
• Change in STI
trends
• Increase in social
support
Long-term effects/
changes in:
• HIV/AIDS trends
• AIDS-related
mortality
• Social norms
• Coping capacity in
community
• Economic impact
• Resources
• Staff
• Funds
• Facilities
• Supplies
• Training
ALL MOST SOME FEW
INPUTS
OUTPUTS
OUTCOMES
IMPACT
Monitoring and evaluation results pyramid
MONITORING
Process evaluation
EVALUATION
Effectiveness evaluation
LEVELS OF EVALUATION EFFORTS
NUMBER
OF
PROJECTS
Introduction
covered elsewhere. Thus, it does not cover
in any detail surveillance, essential
research or financial management. The
role of these elements within an overall
M&E programme is discussed, but users
are referred to sources in Appendix 10 for
detailed information. Thus, the manual
focuses more on tracking inputs and out-
puts than outcomes and impacts.
What is M&E?
11. Confusion between monitoring and
evaluation is common. There is a simple
distinction between monitoring and evalua-
tion that may be helpful. Monitoring is the
routine, daily assessment of ongoing activ-
ities and progress. In contrast, evaluation is
the episodic assessment of overall achieve-
ments. Monitoring looks at what is being
done, whereas evaluation examines what
has been achieved or what impact has been
made.
What key M&E lessons have been
learned?
12. We have learned the following key
M&E lessons:
(i) The results pathway or cycle, shown in
the figure below, may be likened to a
pyramid. The higher up the results
cycle we go, the fewer organizations,
projects and studies are involved in
M&E. Thus, all implementing part-
ners should collect complete input and
output data. Many implementing part-
ners should collect some process data.
Far fewer implementing partners will

4
assess outcomes. Even fewer imple-
menting partners and studies are
required to assess impact.
(ii) Good M&E requires both internal
self-assessment and external verifica-
tion. Thus, implementing partners col-
lect their own internal data and an
external agency verifies the complete-
ness and accuracy of the data collect-
ed by those implementing partners.
Supervisory visits should be based on
the analysis of internal self-assessment
and externally verified primary data.
(iii) M&E systems must be as simple as
possible. Most programmes collect far
more data than they use. The more
complex an M&E system, the more
likely it is to fail.
(iv) M&E systems must include a stan-
dardized core. If each implementing
partner uses different systems or tools,
the data cannot be analysed or sum-
marized effectively. The need for a
standardized core does not preclude
individual implementing partners
from collecting additional, situation-
specific M&E data.
(v) A specialized entity is required to col-
lect, verify, enter and analyse primary
M&E data from each partner. Without
such an entity, data collection, verifi-
cation and analysis are unlikely to
happen. Ministries and other public
agencies are seldom equipped to man-
age such a process.
(vi) Contracting a single specialized entity
to manage both financial and pro-
gramme M&E should be seriously
considered. There are several reasons
why this is desirable, which are listed
below.
• Since the same personnel and oper-
ations can often be used for both
financial and programme monitor-
ing, combining financial and pro-
gramme M&E makes proper moni-
toring economical and affordable.
It may be added to financial man-
agement at a marginal cost.
• Combining financial and pro-
gramme monitoring provides a
basis for cross-checking financial
and activity data and ensuring
sound finance-programme data
cross-verification.
• Given the sensitivities concerning
data verification, it may be prefer-
able to delegate the task to account-
ing/consulting firms that are used
to handling such sensitive opera-
tions.
• Programme M&E entails complex
data-gathering-management capaci-
ties, and it is usually easier to pur-
chase established capacity than to
establish it afresh.
• Financial management monitoring
systems and procedures are almost
always the best developed subcom-
ponent of M&E. It makes sense to
link programme monitoring to the
stronger process of financial moni-
toring.
• Linking programme reporting to
financial management and further
disbursements will improve pro-
gramme reporting.
• Outsourcing financial and pro-
gramme activity monitoring to a
single entity ensures that financial
and programme reports are linked
and provide a more comprehensive
picture. Equipped with comprehen-
sive, verified data, NAC M&E staff
members are free to focus on the
strategic programme implications
of monitoring.
Introduction
Contracting a single specialized entity to
manage both financial and programme
M&E should be seriously considered.

5
Introduction
(vii) M&E must be built into the design of
a programme, and should be opera-
tional before grants are provided,
rather than being added later. It is
much harder and less effective to
'retrofit' M&E after grants have
already been given.
(viii)No matter how sound an M&E system
may be, it will fail without wide-
spread stakeholder 'buy-in'. Thus, a
large-scale, participatory process is
essential to build ownership and buy-
in from the start.
(ix) NACs lack comprehensive, long-term
funding for all major M&E compo-
nents, including local costs and incre-
mental operating costs. The World
Bank, through MAP credits, may pro-
vide comprehensive, long-term M&E
funding in areas where grant funding is
unavailable. The World Bank recom-
mends that up to 10% of MAP credits
be used for local and operating costs of
a long-term M&E system.
(x) It may be more helpful to move from
baseline and follow-up analysis to
trend-tracking, following trends over
several time periods.
What framework is suggested for
M&E?
13. Effective M&E is based on a clear,
logical pathway of results, in which results
at one level are expected to lead to results
at the next level, leading to the achieve-
ment of the overall goal. Consequently, if
there are gaps in the logic, the pathway will
not lead to the required results.
14. The major levels are:
• inputs
• outputs
• outcomes
• impacts
15. These levels are introduced in Panel 1
below. Each level is connected to the next,
in a clear, logical way.
Level Description
Inputs Inputs are the people, training, equipment and resources that we put into a project, in
order to achieve outputs.
Outputs Outputs are the activities or services we deliver, including HIV/AIDS prevention,
care and support services, in order to achieve outcomes.
The processes associated with service delivery are very important and involve qual-
ity, unit costs, access and coverage.
Outcomes Through the provision of good-quality, economical, accessible, and widespread serv-
ices, key outcomes should occur. Outcomes are changes in behaviour or skills, espe-
cially safer HIV prevention practices and increased ability to cope with AIDS.
Impacts The above-mentioned outcomes are intended to lead to major measurable health
impacts, particularly reduced STI/HIV transmission and reduced AIDS impact.
Panel 1: M&E levels
No matter how sound an M&E system may
be, it will fail without widespread stake-
holder ‘buy-in’.

6 Introduction
Component
Overall system
Surveillance
Research
NAC, public sector and civil
society financial management
monitoring
NAC, public sector and civil
society programme activity
monitoring
Description
Overall flowchart and database
National biological and behavioural surveillance of
STI/HIV/AIDS/TB, and sexual behaviour trends.
Essential research to complement national surveillance.
National financial management monitoring of NAC, the
public sector and civil society's utilization of resources.
National programme activity monitoring of NAC grants pro-
vided to implementing agencies, and the relevance, quantity
and quality of services delivered by those agencies.
Panel 2: M&E components
What M&E components exist and
how well developed are they?
16. We suggest the following M&E com-
ponents (see Panel 2 above).
17. M&E strengths in each of the above
components vary widely.
Overall system
18. The overall system comprises a gov-
erning flowchart and database, which
describe precisely how data are collected
and flow.
Surveillance
19. Surveillance comprises biological,
behavioural and social impact surveillance.
Both are well developed globally. WHO,
UNAIDS and CDC support ensures sound
antenatal biological surveillance in most
countries and a proven procedure for estab-
lishing it in other countries. Behavioural
and social impact surveillance, supported
by UNAIDS/FHI and social assessment
guidelines, is widespread and readily
applicable. This surveillance should be
accompanied by second generation surveil-
lance (see Box 1). With such support,
NACs can quickly initiate sound surveil-
lance. In mature epidemics, NACs will
usually use existing surveillance; in nas-
cent epidemics, they may use improved
surveillance. NACs should provide ade-
quate resources and support to National
AIDS Programmes (NAPs) within min-
istries of health to ensure sound surveil-
lance and health-related M&E.
Research
20. Surveillance should be complemented
by essential research, including epidemio-
logical, evaluation and social impact res-
earch. NACs have a strategic role in collat-
ing, interpreting and disseminating res-
earch findings.
Financial management monitoring
21. NAC, public sector and civil society
financial management monitoring is well
supported. The World Bank, for example,
has substantial in-house financial manage-
ment capacity and experience. Social pro-
tection funds have demonstrated the feasi-
bility of outsourcing financial management
to accounting firms/banks.
Programme activity monitoring
22. NAC, public sector and civil society
programme activity monitoring represents
the greatest challenge facing NACs. It is
addressed partly through draft operations
manuals, but significant challenges remain.
NACs will assume a major role as grant-
provider, supporting literally hundreds of

7
Introduction
BOX 1: What is second generation surveillance?
Traditional surveillance systems typically tracked HIV or sexually transmitted infections
(STIs). However, they did not concurrently track the sexual practices that lead to HIV/STI
transmission. This made it difficult to corroborate and explain HIV/STI trends. To address
these limitations, second generation surveillance evolved. This form of surveillance seeks
to combine biological and behavioural data, to increase explanatory power. The concor-
dance of diverse biological, behavioural and qualitative insights not only enhances confi-
dence in trends, but it allows for meaningful explanations of these trends. Examples from
a wide range of countries, including Senegal, Thailand and Uganda, show how second gen-
eration surveillance can identify HIV trends through biological surveillance, and then con-
vincingly explain these trends through behavioural surveillance. Such examples underscore
the vital importance of second generation surveillance.
HIV/AIDS prevention, care and mitigation
activities. However, they lack essential sys-
tems and procedures, particularly those
required for:
• maintaining an overall integrated
M&E flowchart and database;
• identifying epidemiological priorities
and soliciting compliant applications;
• publicizing the availability of funding
for public sector and civil society ini-
tiatives and application mechanisms;
• developing and publicizing structured,
transparent selection criteria and
approval procedures;
• publicizing grant recipients;
• monitoring programme progress of
recipients and communicating achie-
vements; and
• reviewing overall national programme
progress, with particular reference to
geographic focus, coverage and equi-
ty, interventions and service to vulner-
able groups.
23. Programme activity monitoring is
least developed and requires the greatest
emphasis. As with financial monitoring,
this component should be contracted out to
an independent firm. It is recommended
that financial and programme activity
monitoring be combined and contracted
out to one firm, for economy and finance-
programme cross-verification.
24. The best developed components are:
• surveillance (especially biological
surveillance);
• research; and
• financial monitoring.
25. The least developed components are:
• overall M&E system, with flowcharts
and a unified database; and
• programme monitoring.
Who should do what?
26. Building on the first lesson learned (see
the monitoring and evaluation results pyramid
on page 3), data sources, implementing part-
ners' responsibilities for M&E and suggested
time frames for progress are summarized in
Panel 3 (see page 8).
NAC indicators and data sources
27. We suggest an illustrative indicator set,
with data sources, inAppendix 1.This indica-
tor set is neither definitive nor exhaustive.
Individual programmes will both change and
add to this indicator set. However, it does pro-
vide examples of possible indicators and data
sources at each level of M&E.

8 Introduction
Level
Inputs
Outputs
Quantity
Quality
Unit
costs
Access
and
coverage
Outcomes
Impacts
Overall
system
Data
Finance and programme
monitoring
monitoring
Programme monitoring
using quality checklists
monitoring
Modules of behavioural
surveillance and facility
surveys
Behavioural surveillance
and epidemiological
research
Biological surveillance
and epidemiological
research
Flowchart and database
Partner role
All implementing partners submit month-
ly data
Specialized external agency routinely
analyses and verifies data
All implementing partners submit month-
ly data
analyses and verifies data
All implementing partners do internal
quality assurance
does external quality verification
Specialized external agency will use veri-
fied financial and programme output data
to estimate unit costs for selected imple-
menting partners
Access to prevention, care, mitigation
services and coverage will be included as
a subset of behavioural surveillance,
social impact surveys and facility surveys,
and assessed when behavioural or facility
surveys are used
Behavioural surveys to assess outcomes
are encouraged in 5-10 sites per country
every 1-2 years. Behavioural surveys may
also be conducted in selected large-scale
public sector or civil society programmes.
Examples include public sector pro-
grammes for transport workers or soldiers
and civil society programmes for refugees
Behavioural surveys should be contracted
to specialized agencies and use UNAIDS
and FHI guidelines
The ministry of health, often assisted by
WHO, UNAIDS and CDC, is responsible
for national STI and HIV surveillance
Selected epidemiological STI/HIV preva-
lence/incidence studies may also be con-
ducted and may illustrate impacts in spe-
cific areas/populations
NAC maintains overall flowchart and
database
Time frame
Progress within
6 months
Progress within
1 year
Progress within
1-2 years
Progress within
2-3 years
Progress within
3-5 years in
mature epidemics
and 7-10 years in
nascent epidemics
To be designed
before NAC
begins providing
grants
Panel 3: Data sources, partners' roles and time frame by M&E level

9
Overview
28. We propose the following steps for
putting M&E into practice:
(i) NAC clarifies its coordination role
and increases its capacity to coordi-
nate, but not implement, M&E.
(ii) NAC contracts out the implementa-
tion of M&E to specialized entities.
Thus, surveillance, research, and
financial and programme monitoring
should ideally be contracted out to a
range of entities. The specialized pro-
gramme-activity-monitoring entity is
responsible for training implement-
ing partners and verifying, collating,
analysing and reporting data.
(iii) NAC and stakeholders engage in an
intensive participatory process to
build ownership and buy-in, particu-
larly for the overall M&E system and
programme monitoring.
(iv) Each implementing partner agrees on
its key targets with NAC, using a
simple, structured planning, monitor-
ing and evaluation form.
(v) Each implementing partner reports
results every month using the plan-
ning, monitoring and evaluation form.
(vi) These results are checked and veri-
fied at least every six months by the
specialized entity.
(vii) The specialized unit/agency assesses
each implementing partner's progress
towards targets every six months and
rates their progress using the plan-
ning, monitoring and evaluation form.
(viii) The specialized entity collates, analy-
ses and submits to NAC summary
reports of aggregate activities every
six months, using a simple, struc-
tured progress report form.
(ix) NAC and key stakeholders, including
donors, meet every six months to
review M&E reports, to identify key
lessons learned and to make strategic
recommendations and decisions.
(x) NAC and key stakeholders update
their M&E manuals and procedures
based on lessons learned.
Clarifying NAC's role and increasing
its capacity to coordinate
29. A small NAC M&E structure is sug-
gested, with one or two staff (no more are
required for what is purely a coordination
and facilitation role), depending on the size
of the country and the complexity of the pro-
gramme.The M&E staff's proposed terms of
reference are outlined in Appendix 2.
30. A major element of NAC's coordina-
tion role is effective coordination with the
health sector, which is historically the
repository of M&E capacity in AIDS pro-
grammes.
31. There is an urgent need to develop a
shared NAC and donor vision of NAC
M&E philosophy and priorities and to
ensure coordinated and complementary
donor inputs to NAC M&E plans.
Contracting an M&E consultant
32. NAC's role is to ensure that the results
of M&E are used at the appropriate level.
NACs may contract an M&E consultant to
assist with the design and management of
the steps outlined below. Such a consultant
should, above all, be interdisciplinary and
able to combine insights from research, pro-
gramme management and information sys-
tems. Further guidance on suitable M&E
consultants is presented in Box 2.
Operations procedures
B. OPERATIONS PROCEDURES
NAC clarifies its coordination role and
increases its capacity to coordinate, but not
implement, M&E.

10 Operations procedures
Box 2: What is an M&E consultant?
The essence of an M&E consultant is interdisciplinary experience, understanding,
insights and instincts. Above all, the consultant must fully grasp that M&E consists of
several discrete components, all of which must be addressed. Thus, a consultant should
have some epidemiological literacy. However, this may be the least important overall
skill, since epidemiological surveillance is typically better developed than other compo-
nents. Very detailed surveillance procedures exist, and numerous national and interna-
tional organizations can assist with surveillance.
The consultant should, above all, have a firm grounding in programme activity M&E,
because it is the weakest component of the overall M&E system with the least developed
procedures and relatively few experienced practitioners. The consultant should have
managed a project wherein systematic input, output and process data were collected,
summarized and used to guide programming. The consultant should understand the flow
of data, level by level, from a single youth peer educator or grandmother supporting
orphans, to a national information system. Ideally, the consultant should have enough lit-
eracy in management information systems to specify the systems needed and to review
systems commissioned. Combining epidemiological, evaluation, programme and infor-
mation system skills, the consultant should be able to devise an integrated system that
generates and links verified primary biological, behavioural, programme and financial
data. The M&E consultant should be committed to simple, low-cost, low-technology
approaches that may be widely applied in the field.
It is also helpful to emphasize what an M&E consultant should not be. The consultant
should not be an academic researcher concerned with indisputable proof; an epidemiol-
ogist who equates M&E with surveillance; an evaluation expert who equates M&E with
impressionistic external evaluations, often conducted without primary data; or a man-
agement information systems expert who equates M&E with information technology.
33. NACs are strongly encouraged to
recruit an M&E consultant to help build an
overall M&E system. Detailed terms of
reference, an illustrative job advertisement
and scoring criteria are presented in
Appendix 3.
Recruiting specialized M&E units
and agencies
34. In keeping with its coordination role,
it is suggested that NACs outsource M&E
components, as shown in Panel 4 (see
page 11).
35. The suggested terms of reference for
the specialized programme activity moni-
toring unit/agency are presented in
Appendix 4.
Developing an M&E manual with
participatory approaches
36. NAC experience highlights the impor-
tance of participatory approaches to build
ownership and buy-in for M&E. The fol-
lowing participatory process is suggested:
• undertake preparatory research;
• determine interim M&E Reference
Group and indicators;
• organize district and national stake-
holder consultative meetings;
• produce a draft M&E manual;
• organize district and national stake-
holder meetings to review draft M&E
manual;

11
Component
Overall system
Surveillance
Research
National NAC, public sector and civil
society financial management monitoring
National NAC, public sector and civil
society programme activity monitoring
Contracted out to
NAC/Consultant
Biological: National AIDS or epidemiology
programme, supported by surveillance
expert committee
Behavioural: Universities, research agencies
or consulting firms
Universities/institutions
Major accounting/consulting firms
Major accounting/consulting firms
Panel 4: Recommended M&E agencies by M&E component
• finalize M&E manual; and
• organize a national M&E launch
meeting.
Preparatory research
37. Preparatory research to identify exist-
ing M&E approaches, opportunities and
constraints and to identify key issues for
further analysis should be undertaken in
the first month of the exercise. Preparatory
research includes a document review, inter-
views and field visits.
38. The document review should include
the following country-specific documents:
• the National Strategic Plan;
• NAC strategic plans, workplans and
draft operations manuals; and
• ministry of health surveillance and
M&E reports.
39. Key stakeholder interviews should be
undertaken to solicit stakeholders' advice
and concerns regarding M&E. The stake-
holders should include:
• the ministry of health, other key min-
istries and key implementing partners;
• NAC staff and consultants;
• the expanded UN Theme Group mem-
bers, including the UNAIDS Country
Programme Adviser (CPA);
• major bilateral/multilateral donors;
• major NGOs;
• major academic/research institutions;
and
• major groups of people living with
HIV/AIDS (PLWHA).
40. Field visits to a broad cross-section of
stakeholders should be undertaken to inter-
view field staff and beneficiaries and
review existing M&E systems, procedures,
manuals, forms, checklists and reports.
These field visits will provide a communi-
ty perspective and will also yield field
information on what has and has not
worked in M&E. They will also generate
practical examples of effective M&E sys-
tems and tools.
41. Based on the document review, inter-
views and field visits, a synthesis of exist-
ing M&E strengths and gaps and existing
and potential M&E resources should be
prepared.

12 Operations procedures
Interim M&E Reference Group and
indicators
42. An Interim M&E Reference Group
should be formed by NAC within the first
month. This group will meet monthly and
on an ad hoc basis, as required, to provide
advice and to review draft outputs. The
interim group may include the following
representatives:
• ministry of health and other key min-
istries;
• NAC;
• expanded UN Theme Group members,
including the UNAIDS CPA;
• key bilateral/multilateral donors;
• key NGOs;
• academic/research institutions; and
• PLWHA groups.
43. By the second month, NAC, advised
by the Interim M&E Reference Group,
should prepare a set of interim indicators
and instruments to enable NAC to monitor
and evaluate existing projects while a sub-
stantive M&E system is developed.
Without interim indicators and instru-
ments, there is a risk that M&E will not get
under way for several months. The Interim
M&E Reference Group will undertake to
do the following:
• develop a set of interim indicators;
• develop interim data collection tools;
• develop interim data collection proce-
dures;
• institute interim data collection;
• supervise interim data collection; and
• incorporate interim data collection les-
sons into substantive M&E plans.
District and national stakeholder consul-
tative meetings
44. By the third month, a two-day district-
level stakeholder consultative meeting
should be convened to develop a detailed
district M&E strategy, including district
indicators and collection mechanisms.
NACs will ensure balanced representation
from the following public and civil society
stakeholders:
• national and provincial NAC mem-
bers;
• the ministry of health;
• other key ministries, including Educa-
tion and Social Welfare;
• district NGOs and community-based
organizations (CBOs);
• academic/research/consulting groups
with local activities; and
• local PLWHA groups.
45. Also by the third month, a national
stakeholder meeting should be convened to
develop a national M&E strategy, includ-
ing national indicators and collection
mechanisms. NAC will ensure balanced
representation from the following public
and civil society stakeholders:
• the ministry of health and other key
ministries;
• NAC staff and consultants;
• the expanded UN Theme Group mem-
bers, including the UNAIDS CPA;
• major bilateral/multilateral donors;
• major NGOs;
• academic and research institutions of
national relevance; and
• major PLWHA groups.
46. The meeting will develop a draft
M&E strategy to provide a framework for
the development of a draft M&E system.
Draft M&E manual
47. By the fourth month, and based on the
above-mentioned consultative steps, a
small nucleus of NAC members and con-
sultants should develop a draft M&E man-
ual, including:

13
• the overall M&E framework;
• the overall governing flowchart;
• the overall M&E database;
• M&E systems and data collection
instruments; and
• a detailed M&E workplan and budget.
District and national stakeholder consul-
tative meetings to review draft M&E
manual
48. Also by the fourth month, a second
series of (a) district and (b) national stake-
holder consultative meetings should be
convened to carefully review and revise the
draft M&E manual. It will be attended by
the same broad range of constituents iden-
tified above.
Finalization of M&E manual
49. Based on the feedback and revisions
from the district and national meetings and
other sources, a final M&E manual will be
prepared by the fifth month.
National launch meeting
50. By the sixth month, a half-day nation-
al meeting should be held in a major city
(or major cities) to launch the M&E manu-
al. The launch may be attended by up to
200 stakeholders from the public and pri-
vate sectors.
51. An illustrative time frame and budget
for the above-mentioned participatory
process are presented in Appendices 11 and
12, respectively.
Implementing the agreed M&E
system
Monitoring and reporting mechanisms
52. The next stage involves implementing
the agreed M&E system, using the follow-
ing steps:
(i) Each implementing partner refers to
the planning, monitoring and evalua-
tion form in Appendix 5. The reviewed
indicators should be listed in column 1
and targets agreed with NAC for each
relevant indicator in column 2. (Not all
indicators apply to all implementing
partners.)
(ii) Each implementing partner records
their ongoing services on the service
delivery section of the planning, mon-
itoring and evaluation form.
(iii) Every six months, each implementing
partner uses the planning, monitoring
and evaluation form to report their
progress towards agreed targets in col-
umn 2. The specialized agency veri-
fies data every six months.
(iv) During the six-monthly data-verification
visits, the specialized unit/agency
assesses the quality of services deliv-
ered, using quality-assurance check-
lists adapted by NACs for each coun-
try context. Examples of 'programme
areas requiring quality-assurance
checklists' and 'an illustrative quality-
assurance checklist for interpersonal
communication' are presented in
Appendices 6 and 7, respectively.
(v) The specialized agency uses the plan-
ning, monitoring and evaluation form
to rate progress towards targets every
six months, according to the following
simple rating scale:
• Targets largely/completely attained
• Targets partially attained
• Targets largely/completely unattained
(vi) The specialized agency collates,
analyses and submits to NAC summa-
ry reports of aggregate activity every
six months, using the planning, moni-
toring and evaluation form and a struc-
tured progress report form.

14
Programme activity monitoring through
intermediate structures
53. The above-mentioned operational pro-
cedures refer to situations in which the spe-
cialized monitoring unit/agency directly
monitors implementing partners. There are
several situations in which the specialized
entity will work through intermediaries to
monitor implementing partners. Three
examples are presented below:
• provincial intermediaries, in contexts
where NACs devolve major grant-
provision and monitoring responsibili-
ties to provincial bodies;
• district intermediaries, in contexts
ties to district bodies; and
• NGO intermediaries, in contexts
ties to NGOs.
54. How is programme activity monitor-
ing undertaken when intermediary bodies
are involved? The steps listed below are
suggested:
(i) A specialized monitoring entity is still
required, even when elements of mon-
itoring are devolved to districts.
However, the entity's role changes
from direct implementation to capacity-
building and supervision.
(ii) The specialized entity trains and sup-
ports intermediary bodies to equip
them with the skills to undertake pro-
gramme activity monitoring.
(iii) The specialized entity supervises and
monitors intermediary bodies, to ensure
that their monitoring is timely, method-
ical, complete, accurate and in accor-
dance with the procedures manual.
(iv) The specialized entity is still responsi-
ble for collecting, verifying, entering
and analysing primary M&E data from
implementing partners and for prepar-
ing and submitting regular M&E
reports to NAC.
Access database
55. For the overall M&E system, an M&E
database has been written in Microsoft
Access and may be obtained from the
World Bank. The database is written in
Access because most countries have
Access packaged with Microsoft Office,
most consulting firms use it and most local
IT specialists support it.
Checklist for programme managers
56. A checklist to assist programme man-
agers in managing the M&E function is
presented in Appendix 8.
Indicative M&E budget
57. The UNAIDS Monitoring and
Evaluation Reference Group (MERG) is
working on a costing model for an overall
programme. Appendix 9 offers an indica-
tive budget for the M&E components
emphasized in this manual, specifically the
design of an overall M&E system and the
programme activity monitoring compo-
nent. The budget in Appendix 9 does not
include cost estimates for the surveillance,
research or financial management compo-
nents.
58. The indicative budget includes the
costs of designing the overall M&E system
and the programme activity monitoring
system, but it does not include the costs of
actually operating the system. It is possible
to offer tentative guidelines concerning
these costs. The key factor influencing cost
will be whether financial monitoring and
programme monitoring are combined and
delegated to a single entity. Delegating this
activity to a single entity is significantly
cheaper. If this is done, it is estimated that
programme activity monitoring may be
conducted for 2-3% of the total cost of

15
grant funds transferred to partners. If
financial and programme activity monitor-
ing are not combined, but done separately,
it is estimated that programme activity
monitoring will require 5-6% of the total
costs of grant funds transferred to partners.
It should be noted that these percentages
apply only to grant funds transferred to
implementing partners, not to the overall
NAC budget.

Indicator
Prevention
1. HIV and syphilis prevalence among (a) all antenatal women; (b) women aged
15-19; and (c) women aged 20-24
Mitigation
2. Increased quality of life for PLWHA and orphans and other vulnerable chil-
dren (OVC)
Prevention
3. Percentage of respondents who both correctly identify ways of preventing the
sexual transmission of HIV and reject major misconceptions about HIV trans-
mission and prevention
4. Safer sexual practices: youth (15-19)
The (a) increased age of sexual inception and (b) reduced occurrence of unpro-
tected sexual intercourse
5. Safer sexual practices: adults (20-49)
Reduced occurrence of unprotected sexual intercourse with non-regular partner
Mitigation
6. Increased PLWHA/OVC household-coping capacities
Increase NAC capacity
7. NAC board and staff appointed and functional
8. NAC workplans and budgets developed
9. NAC financial, procurement, implementation, technical support and M&E sys-
tems established
10. NAC fund disbursement ratios
11. Number and percentage of districts with HIV/AIDS workplans and budgets
approved and funded
Increase public sector services
12. The (a) number and (b) percentage of line ministries with HIV/AIDS workplans
and budgets for employees
13. The (a) number and (b) percentage of health facilities providing HIV/AIDS care
appropriate for level of facility
14. The (a) number and (b) percentage of primary/secondary/tertiary education
institutions with HIV/AIDS programmes for their students
15. The (a) number and (b) percentage of districts with functioning social welfare
departments providing grants to orphans and other vulnerable children
16. Total AIDS services delivered by public sector
18 Appendices 1 to 9: Tools
APPENDIX 1. Illustrative indicators3
3Although the text presents the results chain in causal order, from inputs to impact, this appendix presents the
results chain in conventional logical framework order, from impact to inputs.
Data source
Antenatal surveillance
Household surveys
DHS, MICS and BSS
and social impact surveys
and social impact surveys
Household surveys
NAC reports
NAC workplans and budgets
NAC reports
NAC reports
NAC reports
NAC reports
Health facility surveys
NAC reports
NAC reports
NAC reports
Outcome level (behavioural outcomes)
Impact level (health impact)
Output level (activities)

19
Appendices 1 to 9: Tools
Increase civil society services
17. Number of civil society organizations receiving NAC funding
18. Percentage of overall funding granted to civil society services
19. Number of new civil society partners introduced to HIV/AIDS programming
with NAC support
20. Total HIV/AIDS services delivered by civil society
HIV/AIDS services: prevention
21. The (a) number of HIV/AIDS radio/television programmes produced and (b)
number of hours aired
22. The number of HIV/AIDS prevention brochures/booklets (a) developed and
(b) distributed
23. The number of (a) HIV prevention staff and (b) volunteers trained
24. The (a) number of HIV prevention meetings held and (b) number of
men/women reached
25. The number of condoms sold/given
26. The number of men/women receiving STI care from health facilities with
trained staff and uninterrupted supply of drugs
27. The (a) number and (b) percentage of men/women receiving HIV testing and
counselling
28. The (a) number and (b) percentage of women tested and receiving PMTCT if
HIV-positive
HIV/AIDS services: care
29. Number of care (a) staff and (b) volunteers trained
30. The (a) number of PLWHA support groups; (b) number of men/women
enrolled; and (c) percentage of men/women enrolled
31. The (a) number of community HIV/AIDS care projects; (b) number of
men/women enrolled; and (c) percentage of men/women enrolled
32. The (a) number of community orphan support projects; (b) number of orphan
boys/girls enrolled; and (c) estimated percentage of orphan boys/girls enrolled
33. The (a) number and (b) estimated percentage of orphan boys/girls receiving
support for school fees
34. Paid staff, volunteers recruited, training conducted, equipment and resources
provided
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC reports
NAC records
Input level (personnel, training, equipment and funds)

APPENDIX 2. Summary terms of reference for NAC M&E staff
• Prepare an overall and annual M&E plan
• Prepare semi-annual and annual M&E reports
• Prepare technical specifications for each M&E component and contract external
agencies to manage each component
• Supervise the quality and timeliness of M&E products contracted out
• Review contracted M&E products and distil and communicate their implications for
programme implementation, including modifications in geographic priorities, target
groups, interventions and implementing partners. M&E must play a central role in
shaping programme direction

21
Background
The National AIDS Council (NAC) was
established in [date] and is recruiting staff.
It is also developing its management sys-
tems, including financial management,
procurement, implementation management
and M&E systems.
A consultant is required for [number of
months], from [date] to [date], to help
NAC develop its M&E system.
Objectives
The key objectives of the consultancy are:
• to help NAC develop an overall M&E
coordination plan, with manuals, sys-
tems, procedures, tools, a database,
flowcharts for data and clearly speci-
fied institutional roles and responsi-
bilities and an implementation plan
and budget; and
• to strengthen NAC's monitoring sys-
tems, to ensure sound output and
process monitoring.
There are existing M&E systems, manuals
and tools prepared for HIV/AIDS pro-
grammes elsewhere in Africa, but these
systems, manuals and tools must be adapt-
ed to the country context, in consultation
with a broad range of country stakeholders.
Selection
The consultant will be selected by NAC, in
consultation with its major partners, and
housed and supported by NAC. The con-
sultant will receive technical support from
NAC and other technical partners.
Skills and experience
The key skills required for the consultancy
are:
• communication and facilitation skills,
to ensure that the M&E system, man-
ual and tools are developed with the
full participation of NAC, govern-
ment, and corporate and civil society
stakeholders;
• coordination skills, to ensure that the
contributions of diverse stakeholders
are effectively harmonized into one
NAC M&E system, with supporting
manual and tools;
• analytical skills, to ensure that the
NAC's M&E system, with supporting
manual and tools, is detailed and logi-
cal; and
• system-building skills, to ensure that
the NAC's M&E system is compre-
hensive, internally consistent, specific
and self-contained, and that it can be
implemented.
Tasks
The consultant's specific tasks are to:
• identify data/reporting needs;
• design the overall M&E system;
• prepare an implementation plan;
• train coordinating and implementing
partners;
• ensure that the system is tested,
refined and fully implemented by
NAC; and
• specify further steps that NAC must
take.
These tasks are outlined in detail below.
1. Identify data/reporting needs
The consultant will build on major activi-
ties already undertaken by NAC. The con-
sultant will rapidly review NAC's existing
logical framework matrix and indicator
sets and propose a circumscribed set of
operational, measurable indicators, with
detailed input, output, process, outcome
and impact measures. The consultant may
APPENDIX 3. Detailed terms of reference for NAC M&E consultant

help NAC streamline its logical frame-
work. Ideally, the logical framework will
not contain more than 40 indicators.
This task will begin on [date] and be com-
pleted, with agreement from all stakehold-
ers, by [date].
2. Design the overall M&E system
The consultant will then help NAC to design
an M&E system and to prepare an M&E
manual. This M&E system should, as far as
possible, reflect the National Strategic Plan.
The consultant will use the current National
AIDS Councils: Monitoring and evaluation
operations manual as a point of departure
when developing NAC's M&E manual.
During preparation of NAC's M&E system
and manual, the consultant will do the fol-
lowing:
• Review documents for the following
components of a comprehensive M&E
system: (i) biological surveillance; (ii)
behavioural surveillance; (iii) research,
especially prevalence, incidence and
intervention impact studies; (iv)
financial management; and (v) pro-
gramme activity.
• Assess how well developed each of
the above M&E components is in the
country context and identify compo-
nents requiring concerted action.
• Collect and include as appendices or
separate volumes all existing proce-
dures manuals that exist for specific
components of NAC's M&E system,
including biological and behavioural
surveillance and clearly defined sub-
components, including blood safety,
PMTCT and VCT.
• Outline a strategic vision and specific
scope of work for M&E in NAC,
which clarifies NAC's coordination
role and increases its capacity to coor-
dinate, rather than implement, M&E.
• Describe precisely how, to whom and
within what period NAC will dele-
gate/contract out each of the compo-
nents and subcomponents of the
national M&E system.
• Present detailed technical specifica-
tions for each component of the over-
all M&E system.
• Outline a participatory process that
NAC may undertake to ensure that
stakeholders are fully involved in the
development of NAC's M&E system,
to ensure ownership and buy-in. The
participatory process will include the
following steps: document review,
stakeholder interviews and field vis-
its; formation of an Interim M&E
Reference Group and indicators; dis-
trict and national stakeholder consul-
tative meetings; preparation of a draft
NAC M&E manual; district and
national stakeholder meetings to
review the draft manual; finalization
of NAC's M&E manual; and a nation-
al meeting to launch NAC's M&E plan
and manual.
• Undertake the participatory process
outlined above to ensure that NAC's
M&E plan is developed in a consulta-
tive manner.
• Prepare a draft NAC M&E manual,
based partly on the current National
AIDS Councils: M&E operations
manual. This manual will reflect the
operational indicators developed for
the logical framework, outline an
overall data collection, collation,
analysis and reporting system and
specify the reporting responsibilities
and interrelationships of all the insti-
tutions involved in M&E at all levels.
The manual will include detailed data
collection procedures, specific flow-
charts to illustrate the precise data
flow from tier to tier, and prototypes
of each of the data collection instru-

23
ments required. It will be prepared so
that it is clear what is required in each
component and at each tier from dis-
trict to national level. It may include
separate modules, which specify as
succinctly as possible what is required
in each component and at each tier.
• Subject NAC's draft manual to inten-
sive review by a broad range of stake-
holders and revise the manual in
accordance with guidance received.
• Finalize NAC's draft M&E manual
based on the above guidance.
• Prepare a flowchart and template for
the development of an Access data-
base to capture all the data required by
NAC's M&E system.
• Liaise with an IT consultant to ensure
that the Access database captures all
the data required by NAC.
• Prepare flowcharts, guidelines and
prototypes for M&E reports from each
of the implementing partners to NAC
and from NAC to parliament.
This component will begin on [date] and
be completed by [date].
3. Prepare an implementation plan
The implementation plan will include a
detailed workplan, time frame, key mile-
stones and budget for each of the M&E
components proposed in NAC's M&E
manual.
4. Train coordinating and implement-
ing partners
The consultant will also train key monitor-
ing partners, particularly NAC, line min-
istries, NGOs and districts, to use the man-
ual to implement effective M&E.
5. Ensure that the system is tested,
refined and fully implemented by
NAC
The consultant will work with NAC to
field-test the entire M&E system. The con-
sultant will revise NAC's M&E system in
accordance with feedback during the field
test. The consultant will then work with
NAC to ensure that the systems are fully
implemented for at least three months
before NAC assumes sole responsibility
for coordinating M&E.
6. Specify further steps that NAC must
take
The consultant will then present a detailed
plan outlining further steps that NAC must
take in order to continue to consolidate
M&E, including the use of mechanisms
whereby NAC can ensure that the manual
is updated through regular feedback and
consultative processes, as required.
NAC may choose to extend the consultancy
to continue implementation beyond the
initial period envisaged in the above-
mentioned points.

Background
A consultant is required for an initial peri-
od of [number of months], with a possible
extension, to develop and test an M&E sys-
tem for a development programme. The
M&E system will be developed in consul-
tation with stakeholders and documented
in a detailed M&E operations manual. The
consultant's major responsibilities are to
facilitate the participatory planning
process, design the overall M&E system
and prepare the M&E operations manual.
The consultant should have good commu-
nication, facilitation, planning and analyti-
cal skills. Applicants should have the fol-
lowing qualifications:
Qualifications
• A master's or doctoral degree or equiv-
alent in a health, social science, man-
agement or engineering discipline and
three years' relevant work experience
OR
• A bachelor's degree in a health, social,
management or engineering sector and
six years' relevant work experience
• Specific experience in facilitating a
participatory planning process for
an activity involving at least 100
staff members or a budget of over
US$1 million
AND
Specific experience in developing
management or M&E systems for an
organization/programme employing at
least 100 staff members or with a
budget of over US$1 million
AND
Specific experience implementing
management or M&E systems for an
organization/programme employing at
least 100 staff members or with a
budget of over US$1 million
• A high level of computer literacy, par-
ticularly in the use of Word, the
Internet and e-mail. Knowledge of
PowerPoint, Excel and Access is a
plus.
Compensation
The position offers attractive remuneration,
working conditions and professional devel-
opment opportunities.
Applications
Candidates are requested to submit a letter
outlining why they are qualified for the
position, and a CV that includes the appli-
cant's telephone number and the telephone
numbers of at least two referees who have
supervised the applicant's professional
work. They should also attach or mail
examples of management or M&E sys-
tems and manuals they have prepared,
indicating their specific contribution to the
product.
Applications may be mailed or submitted
electronically to the address below
[address to be inserted]. Candidates are
encouraged to apply electronically.
APPENDIX 3.1. Draft advertisement for NAC M&E consultant

25
APPENDIX 3.2. Proposed NAC consultant selection scoring criteria
Criteria
Academic qualifications
Relevant professional experience
Computer skills
Total
Weighting
20
60
20
100

APPENDIX 4. Summary terms of reference for specialized programme
activity monitoring entity
• To verify the internal consistency and validity of service delivery data reported by
NAC implementing partners, through at least six-monthly visits
• To cross-validate programme and financial data, and to increase confidence in both
data sources
• To assess the quality of implementing partners' services, using agreed quality-
assurance checklists, through at least six-monthly visits
• To develop a simple management Access database that can be shared widely and used
for further analyses
• To collect, enter and analyse implementing partner programme monitoring data
monthly
• To assist NAC in identifying implementing partners whose performance is exempla-
ry and may serve as a positive example, and implementing partners who are under-
performing, for whom corrective action will be suggested
• To prepare the six-monthly programme monitoring reports, containing summary data,
reviewing overall performance against targets and making overall programme recom-
mendations, including recommendations to improve both programme performance
and M&E

27
Indicator
Prevention
1. HIV and syphilis prevalence among (a) all antenatal women; (b)
women aged 15-19; and (c) women aged 20-24
Mitigation
2. Increased quality of life for PLWHA and OVC
Prevention
3. Percentage of respondents who both correctly identify ways of pre-
venting the sexual transmission of HIV and reject major misconceptions
about HIV transmission and prevention
4. Safer sexual practices: youth (15-19)
The (a) increased age of sexual inception and (b) reduced occurrence of
unprotected sexual intercourse
5. Safer sexual practices: adults (20-49)
Reduced occurrence of unprotected sexual intercourse with non-regular
partner
Mitigation
6. Increased PLWHA/OVC household-coping capacities
Increase NAC capacity
7. NAC board and staff appointed and functional
8. NAC workplans and budgets developed
9. NAC financial, procurement, implementation, technical support and
M&E systems established
10.NAC fund disbursement ratios
11.The number and percentage of districts with HIV/AIDS workplans
and budgets approved and funded
Increase public sector services
12.The (a) number and (b) percentage of line ministries with HIV/AIDS
workplans and budgets for employees
13.The (a) number and (b) percentage of health facilities providing
HIV/AIDS care appropriate for level of facility
14.The (a) number and (b) percentage of primary/secondary/tertiary
education institutions with HIV/AIDS programme for their students
15.The (a) number and (b) percentage of districts with functioning social
welfare departments providing grants to OVC
16.Total HIV/AIDS services delivered by public sector
Rating
of
progress
Progress
towards
targets
Agreed
targets
APPENDIX 5. Planning, monitoring and evaluation form
Impact level (health impact)
Output level (activities)
Outcome level (behavioural outcomes)

Increase civil society services
17.The number of civil society organizations receiving NAC funding
18.The percentage of overall funding granted to civil society services
19.The number of new civil society partners introduced to HIV/AIDS
programming with NAC support
20.Total HIV/AIDS services delivered by civil society
21.The (a) number of HIV/AIDS radio/television programmes produced
and (b) number of hours aired
22.The number of HIV/AIDS prevention brochures/booklets (a) devel-
oped and (b) distributed
23.The number of (a) HIV prevention staff and (b) volunteers trained
24.The number of (a) HIV prevention meetings held and (b) men/women
reached
25.The number of condoms sold/given
26.The number of men/women receiving STI care from health facilities
with trained staff and uninterrupted supply of drugs
27.The (a) number and (b) percentage of men/women receiving HIV test-
ing and counselling
28.The (a) number and (b) percentage of women tested and receiving
PMTCT if HIV-positive
29.Number of care (a) staff and (b) volunteers trained
30.The (a) number of PLWHA support groups; (b) number of
men/women enrolled; and (c) percentage of men/women enrolled
31.The (a) number of community HIV/AIDS care projects; (b) number
of men/women enrolled; and (c) percentage of men/women enrolled
32.The (a) number of community orphan support projects; (b) number of
orphan girls/boys enrolled; and (c) estimated percentage of orphan
boys/girls enrolled
33.The (a) number and (b) estimated percentage of orphan boys/girls
receiving support for school fees
34.Paid staff, volunteers recruited, training conducted, equipment and
resources provided
Input level (deliver personnel, training, equipment and funds)

29
21. Number of HIV/AIDS radio/television programmes
produced and number of hours aired
22. Number of HIV/AIDS prevention brochures/booklets
developed and numbers distributed
23. Number of HIV prevention staff and volunteers trained
24. Number of HIV prevention meetings held and
men/women reached
25. Number of condoms sold/given
26. Number of health facilities providing STI care with
both trained staff and uninterrupted supply of drugs
27. Number and percentage of men/women receiving HIV
testing and counselling
28. Number and percentage of women tested and receiving
PMTCT if HIV-positive
29. Number of care staff and volunteers trained
30. Number of PLWHA support groups and number and
percentage of men/women enrolled
31. Number of community HIV/AIDS care projects and
number and percentage of men/women enrolled
32. Number of community orphan support projects and
number and percentage of boys/girls enrolled
33. Number and percentage of boys/girls receiving support
for school fees
Radio programmes produced:
Television programmes produced:
Number of hours radio programmes aired:
Number of hours television programmes aired:
Number of HIV/AIDS prevention brochures
developed:
Number of HIV/AIDS prevention brochure
distributed:
Staff trained:
Volunteers trained:
Meetings held:
Men reached:
Women reached:
Condoms sold:
Condoms given:
Number:
Number men:
Number women:
Percentage men:
Percentage women:
Number:
Percentage:
Staff:
Volunteers:
PLWHA groups:
Number men enrolled:
Number women enrolled:
Percentage men enrolled:
Percentage women enrolled:
Care projects:
Number men enrolled:
Number women enrolled:
Percentage men enrolled:
Percentage women enrolled:
Orphan projects:
Number boys enrolled:
Number girls enrolled:
Percentage boys enrolled:
Percentage girls enrolled:
Number boys:
Number girls:
Percentage boys:
Percentage girls:
APPENDIX 5.1. Service delivery sub-section

30
Programme area
Prevention:
Mass communication
Interpersonal communication
Condom distribution and promotion
STI care
HIV counselling and testing
Blood safety
PMTCT
Care:
PLWHA support
Clinical AIDS care
Community HIV/AIDS care
Orphans and other vulnerable children
APPENDIX 6. Programme areas requiring quality-assurance checklists

31
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
No
Quality-assurance questions
Was the meeting place as cool and airy as possible?
Was the audience sitting comfortably on seats or mats, for indoor meetings,
or sitting or standing under shade, for outdoor meetings?
Was the audience arranged in a horseshoe, within 5 metres of the presen-
ter(s) for lectures or discussion and within 7 metres for participatory activi-
ties?
Were there at least 10 people in the audience, excluding the presenter/facil-
itator(s)?
Did the presenter/facilitator(s) talk loudly enough for the audience to hear?
Did the audience listen quietly, when the presenter/facilitator(s) spoke, and
were disruptive or drunk people silenced?
If there was a lecture, was it no longer than 10 minutes?
Was there at least one participatory activity, followed by a discussion?
Did the audience show enthusiasm during the participatory activity?
Was all the factual information that was presented in the lecture, participa-
tory activity or discussion, accurate and up to date?
Did the lecture and/or participatory activity avoid blaming women for the
spread of HIV/STIs?
Were there at least 20 minutes, preferably 30 minutes, for discussion?
Did at least 5, preferably 10, members of the audience join in the discus-
sion?
Was the number of women contributing to the discussion proportionate to
the number of women in the audience?
During the discussion, did the presenter/facilitator(s) listen to each comment
without showing facial disapproval or interrupting (except where the speak-
er was drunk or deliberately disruptive)?
Did the presenter/facilitator(s) respond very briefly to each comment, ask-
ing the audience to comment further, without answering the comment per-
sonally?
Did the presenter/facilitator(s) lead the discussion away from basic facts
about HIV/AIDS to attitudes, values and personal concerns?
When women and HIV/AIDS/STIs were discussed, did the presenter/facili-
tator(s) guide the audience to focus on men's responsibility?
Did the presenter/facilitator(s) offer condoms at the end of the meeting?
Did the presenter/facilitator(s) end by telling the audience where and when
they could contact the project officials for further information and refer the
audience for other requested services, such as STI care or VCT?
Yes
APPENDIX 7. Illustrative quality-assurance checklist for interpersonal
communication

No
Yes
Step
Does the project manager possess, or has the project manager contracted
out, sufficient M&E expertise to provide effective oversight?
Has the project manager reviewed the Programme Description Summary
to ensure that its logic is consistent with the M&E manual?
Has the project manager reviewed existing national M&E activities, iden-
tifying which components are already addressed, and which need to be fur-
ther addressed?
Has the project manager ensured that NAC fully understands, and is
unequivocally committed to, coordination and not implementation?
Does NAC have an M&E plan and is it sufficiently detailed, operational
and time-bound to ensure that comprehensive and timely M&E will occur?
Does NAC have sufficient internal, or contracted, capacity to coordinate
M&E?
Has NAC contracted out each M&E component to specialized agencies?
Do contracted agencies have the capacity to conduct the required M&E
activities on time?
Are terms of reference and technical specifications for each monitoring
component clear and comprehensive?
Is there an adequate participatory process to ensure national engagement
and ownership of the overall M&E plan?
Is each specialized agency submitting high-quality M&E inputs, on time,
as specified in the M&E plan and contracts?
If not, is NAC taking prompt and adequate corrective action?
Are NAC and partners meeting as agreed in the workplan to review per-
formance?
Is the Planning, monitoring and evaluation form being completed com-
prehensively and on time?
Is M&E being used to make prompt and appropriate project adjustments?
Are M&E manuals and procedures being updated on the basis of pro-
gramme learning?
APPENDIX 8. Checklist for project managers in NACs and donor agencies
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

33
Activity
International consultant to develop frame-
work and assist in identifying and support-
ing local M&E consultant
National consultant to design overall M&E
system and programme activity monitoring
manual and system
National IT specialist to adapt Access data-
base for national use
Participatory process to ensure stakeholder
commitment and understanding
Total
Budget
20 days x US$700 daily for international
travel, accommodation, meals, local trans-
port and communication and fees
= US$14,000
Subtotal = US$14,000
150 days x US$350 daily for accommoda-
tion, meals, local transport and communi-
cation and fees = US$52,500
15 days x US$300 daily for accommoda-
tion, meals, local transport and communi-
cation and fees = US$4,500
Subtotal = US$4,500
See detailed budget in Appendix 12, which
does not include national consultant fees
budgeted within 150 days above
US$131,000
APPENDIX 9. Indicative budget for design of overall M&E system and
programme activity monitoring
Note: This budget was prepared in 2001 with US$ and is an average figure. It needs to be
adjusted for individual contexts.

APPENDICES 10 TO 12:
FURTHER INFORMATION

36 Appendices 10-12: Further information
APPENDIX 10. Key sources for further information on M&E
The major sources for guidelines cited below are UNAIDS, WHO, MEASURE and FHI.
The latest versions of these guidelines may be found on the Internet at:
http://www.unaids.org
http://www.who.int
http://www.cpc.unc.edu/measure
http://www.fhi.org
http://www.cdc.gov
http://www.usaid.gov
Family Health International (2002) Evaluating Programs for HIV/AIDS Prevention and Care
in Developing Countries: A Handbook for Program Managers and Decision Makers.
Washington: Family Health International.
(http://www.fhi.org/en/aids/impact/impactpdfs/evaluationhandbook.pdf)
UNAIDS/MEASURE (2000) National AIDS Programmes: A Guide to Monitoring and
Evaluation. Geneva: UNAIDS.
(http://www.cpc.unc.edu/measure/guide/guide.html)
Family Health International (2000) Behavioural Surveillance Surveys (BSS): Guidelines for
Repeated Behavioural Surveys in Populations at Risk for HIV. Arlington: Family Health
International.
(http://www.fhi.org/en/aids/wwdo/wwd12a.html#anchor545312)
Centers for Disease Control and Prevention (2002) Strategic Monitoring and Evaluation: A
Draft Planning Guide and Related Tools for CDC GAP Country Programs. Atlanta: Centers for
Disease Control and Prevention.
UNAIDS/Family Health International (2000) Second Generation Surveillance for HIV: The
Next Decade. Geneva: UNAIDS.
(http://www.who.int/emc-documents/aids_hiv/docs/whocdscsredc2005.PDF)
UNAIDS/WHO (1999) Guidelines for Sexually Transmitted Infections Surveillance.
UNAIDS/WHO Working Group on Global HIV/AIDS/STI Surveillance. Geneva: UNAIDS.
(http://www.unaids.org/publications/documents/impact/std/JC240-SexTransmInfSurv-E.pdf)
UNAIDS (1999) Acting Early to Prevent AIDS: The Case of Senegal. Geneva: UNAIDS.
(http://www.unaids.org/publications/documents/epidemiology/determinants/una99e34.pdf)
UNAIDS (1998) The Relationship of HIV and STD Declines in Thailand to Behavioural
Change. Geneva: UNAIDS.
UNAIDS (1998) A Measure of Success in Uganda. Geneva: UNAIDS.
UNAIDS/Family Health International (1998) Meeting the Behavioural Data Collection Needs
of National HIV/AIDS and STD Programmes. Geneva: UNAIDS.
(http://www.fhi.org/en/aids/impact/imppub/bdcbiback.html#anchor1086792)

37
APPENDIX 11. Illustrative time frame for participatory planning process
Month
6
Month
5
Month
4
Month
3
Month
2
Month
1
Action
Interim M&E Reference Group and
indicators
District and national stakeholder meet-
ing to develop draft M&E manual
Development of draft M&E manual
District and national stakeholder
meeting to review draft M&E manual
Finalization of M&E manual
National meeting to launch M&E
manual
Appendices 10-12: Further information
X
X X
X
X
X
X
X

38
Activity
Interim M&E Reference Group and indicators
District and national stakeholder consultative
meeting to develop manual
Development of draft manual
District and national stakeholder consultative
meeting to review draft manual
Finalization of manual
National meeting to launch manual
Consultancy support
Total
Appendices 10-12: Further information
APPENDIX 12. Illustrative budget for participatory planning process
Budget
Monthly meetings: 6 meetings @ US$1,000
per meeting = US$6,000
Subtotal = US$6,000
Facilitation: US$3,000
Consumables: US$2,000
Transport and 2 days’ accommodation: 20
people @ US$400 = US$8,000 x 2 (district
and national)
Facilitation: US$3,000
Consumables: US$2,000
Transport and 2 days’ accommodation: 20
people @ US$400 = US$8,000 x 2 (district
and national)
M&E guide: US$7,000
Transport and refreshments for 200 partici-
pants: 200 people @ US$25 = US$5,000
100 person days @ US$300 = US$30,000
US$90,000
Note: This budget was prepared in 2001 with US$ and is an average figure. It needs to be adjust-
ed for individual contexts. The budget is only for the participatory component of the over-
all M&E planning process.

The Joint United Nations Programme on HIV/AIDS (UNAIDS) is the leading advocate for global action
on HIV/AIDS. It brings together eight UN agencies in a common effort to fight the epidemic: the United
Nations Children’s Fund (UNICEF), the United Nations Development Programme (UNDP), the United
Nations Population Fund (UNFPA), the United Nations International Drug Control Programme (UNDCP),
the United Nations Educational, Scientific and Cultural Organization (UNESCO), the World Health
Organization (WHO), the International Labour Organization (ILO) and the World Bank.
UNAIDS both mobilizes the responses to the epidemic of its eight cosponsoring organizations and
supplements these efforts with special initiatives. Its purpose is to lead and assist an expansion of the
international response to HIV on all fronts: medical, public health, social, economic, cultural, political
and human rights. UNAIDS works with a broad range of partners—governmental and NGO, business,
scientific and lay—to share knowledge, skills and best practice across boundaries.
Produced with environment-friendly materials

The National AIDS Councils: Monitoring and Evaluation Operations Manual is
designed as a practical toolkit and road map for practitioners to use in designing
and implementing programme monitoring and evaluation (M&E). The manual
introduces key concepts; presents simple, clear procedures, with a checklist of the
process, timing and costs of building participatory programme M&E for National
AIDS Councils (NACs); and offers key tools that implementing partners need for
M&E. The manual emphasizes the development of the overall M&E system, in
relation to the National Strategic Plan, and the monitoring of services provided
through NACs and their implementing partners.
The World Bank
1818 H Street, NW
Washington, DC, 20433 USA
Telephone: (202) 473 1000
Fax: (202) 477 6391
E-mail: books@worldbank.org
Internet: http://www.worldbank.org
UNAIDS
20 avenue Appia
1211 Geneva 27, Switzerland
Telephone: (+41) 22 791 36 66
Fax: (+41) 22 791 41 87
E-mail: unaids@unaids.org
Internet: http://www.unaids.org

Monitoring_and_Evaluation_2.pdf

More Related Content

Similar to Monitoring_and_Evaluation_2.pdf (20)

More from ssusere0ee1d (10)

Recently uploaded (20)

Monitoring_and_Evaluation_2.pdf