Improving Data for Decision-Making: Leveraging Data Quality Audits in Haryana, India

IMPROVING DATA FOR DECISION-MAKING:
LEVERAGING DATA QUALITY AUDITS
IN HARYANA, INDIA
May 2014
DISCLAIMER
The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International
Development (USAID) or the United States Government.
This publication was produced for review by the United States Agency for International Development and the Haryana National Rural
Health Mission. It was prepared by Gajinder Pal Singh, Jordan Tuchman, and Michael P. Rodriguez of the Health Finance and Governance
Project.

The Health Finance and Governance Project
USAID’s Health Finance and Governance (HFG) project will improve health in developing countries by expanding
people’s access to health care. Led by Abt Associates, the project team will work with partner countries to
increase their domestic resources for health, manage those precious resources more effectively, and make wise
purchasing decisions. As a result, this five-year, $209 million global project will increase the use of both primary
and priority health services, including HIV/AIDS, tuberculosis, malaria, and reproductive health services. Designed
to fundamentally strengthen health systems, HFG will support countries as they navigate the economic transitions
needed to achieve universal health care.
May 2014
Cooperative Agreement No: AID-OAA-A-12-00080
Submitted to: Scott Stewart, AOR
Office of Health Systems
Bureau for Global Health
And
Ekta Saroha, Project Management Specialist
Strategic Information and Policy
Health Office
USAID India
Recommended Citation: Singh, Gajinder Pal, Jordan Tuchman, and Michael P. Rodriguez. May 2014. Improving
Data for Decision-making: Leveraging Data Quality Audits in Haryana, India. Delhi, India: Health Finance and
Governance Project, Abt Associates.
Abt Associates Inc. | 4550 Montgomery Avenue, Suite 800 North | Bethesda, Maryland 20814
T: 301.347.5000 | F: 301.652.3916 | www.abtassociates.com
Broad Branch Associates | Development Alternatives Inc. (DAI) | Futures Institute

| Johns Hopkins Bloomberg School of Public Health (JHSPH) | Results for Development Institute (R4D)

| RTI International | Training Resources Group, Inc. (TRG)

IMPROVING DATA FOR DECISION-
MAKING: LEVERAGING DATA QUALITY
AUDITS IN HARYANA, INDIA
DISCLAIMER
The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency
for International Development (USAID) or the United States Government.

CONTENTS
Acronyms................................................................................................................. iii

Executive Summary .............................................................................................. vii

1. Introduction ......................................................................................................... 1

2. Data Quality Audit Findings .............................................................................. 9

3. Recommendations and Next Steps................................................................. 17

Annex 1: DQA Protocol 1 Questions.................................................................. 23

List of Tables
Table 1: Verification Factor for DQA Indicators..........................................................................ix

Table 2: Site Visit Schedule..................................................................................................................8

Table 3: Assessment of Data Management and Reporting Systems.........................................9

Table 4: Trace and Verify: Institutional Deliveries to ANC Registrations ...........................13

Table 5: Trace and Verify: Newborns Weighing Less than 2.5 kg to Total Newborns....14

Table 6: Trace and Verify: Cases of Pregnant Women with Complications and Attended

at Public Facilities to Reported Deliveries..................................................................14

Table 7: Trace and Verify: Postpartum Sterilizations to Total Sterilizations.......................15

List of Figures
Figure 1: Spider Graph of Protocol 1 Summary ......................................................................... viii

Figure 2: NRHM Indicators in Relation to the RMNCH+A Continuum of Care.................1

Figure 3: Health Management Information System Data Flow in Haryana.............................3

Figure 4: The Five Domains of the Monitoring and Evaluation System...................................5

Figure 5: NRHM Classification of Districts in Haryana................................................................7

i

ACRONYMS

ANC Antenatal Care
CHC Community Health Centre
DHIS District Health Information System
DQA Data Quality Audit
HFG Health Finance and Governance project
HIS Health Information System
HMIS Health Management Information System
M&E Monitoring and Evaluation
MCTS Mother and Child Tracking System
MOHFW Ministry of Health and Family Welfare
NHSRC National Health Systems Resource Centre
NRHM National Rural Health Mission (now known as National Health Mission)
PEPFAR President’s Emergency Plan for AIDS Relief
PHC Primary Health Centre
RMNCH+A Reproductive, Maternal, Newborn, Child and Adolescent Health
RDQA Routine Data Quality Assessment
TT Tetanus Toxoid
USAID United States Agency for International Development
iii

ACKNOWLEDGEMENTS
The success of this data quality audit exercise was made possible by the support of Haryana State National Rural
Health Mission (NRHM), Mission Director Dr. Rakesh Gupta, NRHM Director Dr. Ravi Kant Gupta, the Project
Officer for Information Technology Mr. Harish Bisht, and the District Civil Surgeons and District Monitoring and
Evaluation Officers in the districts of Bhiwani, Narnaul, Mewat, and Palwal. The Health Finance and Governance
project gratefully acknowledges the assistance provided by the NRHM teams at both the state and district levels in
coordinating and confirming the site visits to health facilities for the audit.
v

EXECUTIVE SUMMARY

The Government of India has initiated a Call to Action intended to accelerate progress toward attaining
Millennium Development Goals 4 and 5: reduction of under-five mortality and improvement of maternal
health outcomes. The government has prioritized 184 of the 640 districts in the country for focused
maternal and child health interventions under an integrated program called the Reproductive, Maternal,
Neonatal, Child and Adolescent Health (RMNCH+A) initiative. A key factor in the success of this
initiative is the ability of the government to effectively track health outcomes through the routine
collection of data from service delivery points across the high-priority districts.
The National Rural Health Mission (NRHM)1 is responsible for monitoring RMNCH+A indicators across
the country and has leveraged the rollout of a web-based national health management information
(HMIS) since 2008 for this purpose. In several states, another information system – the web-based
District Health Information System (DHIS) 2.0 – is used to compile routine data up to the state level,
where the data are then uploaded separately to the national HMIS portal. A number of reviews of the
data produced through the national HMIS have indicated that there are data quality issues. However,
there are limited reviews of data quality taking place across the NRHM facilities and no systematic
assessment mechanism is currently in place.
To address these issues, the Haryana State NRHM partnered with the USAID-funded Health Finance
and Governance Project (HFG) to conduct a data quality audit (DQA) across four of the state’s high-
priority districts using a methodology and set of tools intended to assess underlying systems and
structures supporting the flow of health data. The DQA exercise took place from 4 – 13 December
2013, beginning with a presentation of the methodology to a cadre of Haryana NRHM Program
Managers, HMIS Officers, and District Monitoring and Evaluation (M&E) Officers. The DQA
methodology used was comprised of two components (Protocol 1 and Protocol 2), which looked at the
systems in place to support data reporting and the factors impacting the validity and timeliness of data
submissions, respectively.
The DQA exercise was conducted by HFG as an audit using a purposive sample of 10 health facilities
across the four districts of Bhiwani, Narnaul, Mewat, and Palwal. At each of the facilities visited, the
DQA team examined the systems in place to support data reporting and validated the accuracy and
timeliness of reporting of four indicators with multiple data elements monitored by NRHM to track
RMNCH+A progress. In addition, discussions with the Haryana NRHM M&E staff provided insights into
the reporting of data into the national HMIS web portal after initial compilation of data in the District
Health Information System 2.0 (DHIS 2.0). The findings from the qualitative components of the DQA
exercise are visually summarized in the spider graph shown below, which resulted from data gathered
using Protocol 1. The spider graph, where a 3 is the best possible score and a 1 the lowest, shows that
the Data Management Processes were the weakest overall component evaluated with a composite score
of 1.90, while Monitoring and Evaluation Structure, Functions and Capabilities scored the highest with a
2.74 rating.
1 NRHM has been merged with the National Urban Health Mission to be known as the National Health Mission; NRHM is
maintained throughout the report as it reflects the status of the agency during the study period.
vii

Figure 1: Spider Graph of Protocol 1 Summary
Presented here is a brief summary of the findings and recommendations, sorted by the five domains
evaluated in the DQA exercise:
M&E Structure, Functions, and Capabilities
 There is a clear assignment of duties for data management and reporting within health facilities,
including the review of data for consistency prior to submission into the DHIS 2.0, although there
are no formal written job descriptions for Information Assistants.
 Health facilities are consistently staffed with and well-supported by a cadre of Information Assistants
who have been trained on the use of the DHIS 2.0.
 Once data are entered into the DHIS, there is very little feedback to health facility staff on the
impact, quality or utility of the data being reported. The absence of a routine monitoring and
feedback loop leads to a disconnect between the data compilers and the data being reported.
 Data are rarely being used by health facilities to track their own progress over time and/or to
engage with their target populations to better meet their catchment area’s needs.
Indicator Definitions and Reporting Guidelines
 While written guidelines and data definitions have been compiled and distributed across districts,
copies of the written guidelines are not available to the staff recording routine data at the health
facilities and few of them have been formally trained on the indicators.
 For some indicators (e.g., ‘newborns less than 2.5 kg’ and ‘women with obstetrical complications’),
unclear definitions directly led to inaccurate data being reported. In one case it resulted in the
viii

-
wrong indicator being reported as the units counted were ‘pills distributed’ rather than the ‘number
of women receiving the pills.’
 For the indicators reviewed, there are mixed results with data validity. Using a verification factor
that reflects over-reporting (greater than 1.0), under-reporting (less than 1.0) or an accurate match
(exactly 1.0) between counted and reported, the following were the overall data validation results:
Table 1: Verification Factor for DQA Indicators
Facility Name
Institutional
Deliveries to
ANC
Registrations
Newborns
Weighing less
than 2.5kg to
Newborns
Weighed at Birth
Cases of
pregnant women
with obstetric
complications
and attended at
public facilities to
reported
deliveries
Post Partum
Sterilization as a
ratio to Total
Female
Sterilization
Bhiwani District
Hospital
1.00 1.00 1.00 1.00
CHC Loharu 1.00 1.00 1.00 1.00
CHC Tosham 1.00 0.65 N/A 1.13
Narnaul District
Hospital
0.99 0.97 1.00 0.95
CHC Kanina N/A N/A 1.00 1.00
CHC Nangal
Choudhary
1.00 0.38 1.00 1.00
Mewat District
Hospital
1.00 0.53 0.61 1.00
CHC Ferozpur
Zirka
0.98 1.05 N/A N/A
CHC Nuh 0.88 0.42 2.82 N/A
CHC Hathin 1.07 1.00 N/A N/A
ix

Data Collection and Reporting Forms / Tools
 The use of the DHIS 2.0 for reporting purposes streamlines and simplifies the data aggregation
process within districts and across the state.
 All facilities visited have sufficient information technology infrastructure in place to support the
routine reporting of data, including working computers and regular access to the Internet.
 Although the DHIS 2.0 is uniformly being used for reporting data, there is a wide variation among
health facilities with regard to the source documents being utilized. Standard, pre-printed registers
for data collection are not available to most health facilities, resulting in multiple reporting formats
and increasing the possibility of data reporting errors.
Data Management Processes
 Staff members responsible for collecting and recording data to the source documents (i.e. individual
patient records and/or consolidated registers) are performing well given the absence of formal
training and notable resource constraints.
 Reporting deadlines are widely known and consistently followed across most health facilities visited.
However, the timeliness of reporting submission from the health facilities could not be documented
during the DQA exercise due to the absence of DHIS 2.0 generated reports showing which facilities
submitted their reports on time for the review period. This functionality does exist within the DHIS
2.0 but is not being effectively utilized.
Links with National Reporting System
 Due to the lack of compatibility between the DHIS 2.0 and the national HMIS web portal, extensive
data management activities must take place at the state level in order to comply with the GoI
reporting requirements.
 The existence of multiple information systems at the health facility level (i.e. DHIS and Mother and
Child Tracking System) leads to duplicative data entry and extra workloads on data reporting staff.
RECOMMENDED WAY FORWARD
 The distribution to all health facilities of written indicator definitions, data reporting guidelines, and
standardized RMNCH+A registers (in both English and Hindi) will likely improve data consistency.
 Development and implementation of a routine data reporting training plan will likely increase the
understanding and knowledge of health facility staff on the indicators they are reporting.
 Establishment of a routine data feedback mechanism by the Haryana NRHM to all facility staff
involved in the collection, recording, and compilation of facility data can improve the likelihood that
data will be used by the health facilities. Implementation of the data feedback mechanism could be
rolled out with the routine training discussed above.
 Implementation of a routine data quality assessment process within the existing NRHM supportive
supervision structure by the District M&E Officers may facilitate regular dialogue on data and open
opportunities for data usage.
x

 Recognizing that this effort is beyond the state NRHM controls, the development and
implementation over the long-term of a nationally integrated electronic health information system
based on unique patient identifiers, unique health facility identifiers, and patient-level data captured
on a daily basis may eliminate some of the duplicative reporting and the time associated with those
processes. The GoI has the resources, technology, and skills to implement such a system with the
appropriate level of prioritization from the national level.
Overall, the Haryana NRHM health facilities are leveraging limited resources to routinely compile and
report required data. Data-gathering burdens on service delivery staff are very high given the volume of
indicators required to be reported and the lack of staff dedicated exclusively to the recording of facility
data. Data quality can likely be improved with the implementation of several key interventions requiring
a moderate increase in resources. The opportunity exists for Haryana NRHM to significantly improve
the reporting of health data so that indicators tracking progress toward the key Millennium
Development Goals of reduced maternal and infant mortality are more readily available and trusted to
inform decision-making around improving the impact of the RMNCH+A initiative.
xi

1. INTRODUCTION

The Government of India’s strategic approach to the Reproductive, Maternal, Newborn, Child and
Adolescent Health (RMNCH+A) initiative is based on the central tenets of equity, universal access to
care, entitlement, and accountability. Furthermore, it envisions a comprehensive continuum of care with
integrated service delivery at various life stages: adolescence, pre-pregnancy, childbirth and the postnatal
period, childhood, and through the reproductive ages. Within each of the RMNCH+A program areas, a
set of priority interventions shown to have a notable impact are being rolled out across India, with
emphasis on a set of 184 high-priority districts with high maternal and child morbidity and mortality
indicators. The National Rural Health Mission (NRHM) has been tracking a set of 16 indicators that
measure the impact of these interventions and is promoting the use of a dashboard based on data
generated by the Health Management Information System (HMIS). The dashboard seeks to improve
state accountability for RMNCH+A indicators and catalyze states into using the HMIS data for improved
decision-making. Figure 2 shows the indicator set in relation to the RMNCH+A continuum of care
structure.
Figure 2: NRHM Indicators in Relation to the RMNCH+A Continuum of Care
Note: ANC = antenatal care, IFA = iron folic acid, TT = tetanus toxoid, SBA = skilled birth attendant, IUD = intrauterine device
1

Monitoring of the RMNCH+A strategy is not solely based on data from the national HMIS. A set of 19
outcome and service coverage indicators related to health, nutrition, and sanitation are drawn from
various national and subnational surveys and are supposed to be reviewed on an annual basis. Although
much of the data from these surveys are captured by the HMIS (e.g. maternal mortality rates and
immunization coverage) concerns with HMIS data quality prompt the utilization of alternative data
sources for program planning and decision-making. The District Health Information System (DHIS) 2.0 is
used by health facilities to report their data electronically up to the State NRHM level; district-level
information (i.e. all health facilities within a district) is automatically aggregated within the system. At the
state level the NRHM team is using DHIS 2.0 monthly data to create the RMNCH+A dashboard.
However, the Information Assistants at the facilities are also required to export their DHIS 2.0 data into
a Microsoft Excel format and then upload it into the national HMIS web portal in order to meet the
national NRHM reporting requirements.
In addition, health facilities are responsible for capturing patient-specific data within the Maternal and
Child Tracking System (MCTS) to track care of pregnant mothers and children. Much of the information
captured in the MCTS could also be used to track the RMNCH+A indicators, yet there is no connection
(electronic or otherwise) between the MCTS, DHIS 2.0 or the national HMIS. The presence of multiple
systems results in significant levels of duplicate data entry at the health facility level and a heavy
reporting burden on both the Auxiliary Nurse Midwives and the Information Assistants, who are
involved in recording and reporting the data, respectively, as represented in the data flow diagram
provided in Figure 2 on the next page.
2

-
-
Figure 3: Health Management Information System Data Flow in Haryana
STEPIII
STEPII
STEPI
Primary Health Centre
(PHC and linked HSC)
Parallel Monthly Data Entry in the HMIS Web Portal of
Ministry of Health and FamilyWelfare (MoHFW)
Monthly Data Entry in DHIS 2.0
Haryana Reporting Portal
Health Sub Centres (HSC)
(Primary Data from Multiple Paper Registers,
Uploaded electronically)
Community Health Centre (CHC)
Sub Divisional Hospital
(SDH)
District Hospital
(DH)
STEP I: Facility Wise Data uploaded to the DHIS 2.0 and to the Mother and Child Tracking
System (MCTS)
STEP II: Data downloaded in MS Excel from DHIS 2.0 in MoHFW format for uploading to
national HMIS portal. As there is no electronic bridge between DHIS 2.0 and HMIS, each facility
downloads the monthly data in MoHFW format in MS Excel from DHIS 2.0 every month.
STEP III: Each facility uploads data (exported from DHIS 2.0 in MS Excel format) to the
MoHFW HMIS portal every month by the 5th
of the following month of reporting.
3

The Health Finance and Governance (HFG) project, funded by USAID, has been tasked with supporting
six states and territories in India, particularly the RMNCH+A high-priority districts, to strengthen the
domains of health finance, human resources for health and health information systems (HIS). A key
domain within HIS that HFG is supporting is in improving the production and usage of high-quality health
data. At meetings in October 2013, the HFG and Haryana NRHM teams began discussions about
concerns that NRHM had with the quality of data being reported through the state-sponsored DHIS 2.0
and agreed to develop a work plan to implement a data quality audit (DQA) across key districts in
Haryana. This was followed on 4 December 2013 by a workshop in Panchkula with the district health
teams and state NRHM program officers to share the DQA methodology and to develop a timeline for
field visits. The field visits took place during the week of 6 December 2013 and the initial phase of the
DQA exercise concluded with a debriefing in Panchkula for the Mission Director NRHM’s
representative on 13 December 2013.
DQA METHODOLOGY
Reviews in the spring of 2013 by the National Health Systems Resource Centre (NHSRC) in Haryana
noted that there was routine over-reporting of data across many districts in Haryana State. For
example, looking at data reported through the DHIS 2.0 for the period of April 2012–March 2013,
NHSRC found that the Palwal District reported that antenatal care (ANC) registrations were 47
percent higher than the total number of expected deliveries; reported deliveries were 11 percent higher
than expected deliveries; and measles vaccines administered were 16 percent higher than the number of
live births reported. Overall, NHSRC found that only one district (out of 21 in Haryana) did not have
reported occurrences (e.g. immunization rates, deliveries, children weighed) higher than their population
totals.2
An effective means to evaluate the validity of data generated by a routine HIS in compiling and reporting
health information system is to perform a DQA. The DQA methodology, as implemented in Haryana by
HFG India, was designed to accomplish the following tasks:
 Verify that appropriate data management systems are in place in Haryana;
 Verify the quality of reported data for key RMNCH+A indicators at selected sites; and
 Contribute to monitoring and evaluation (M&E) systems strengthening and capacity building for
NRHM.
The DQA methodology used in Haryana is based on two distinct protocols built into a Microsoft Excel
template; the methodology was first piloted in mid-2006 by a group of international partners including
the President’s Emergency Plan for AIDS Relief (PEPFAR), the United States Agency for International
Development (USAID), the World Health Organization, the Global Fund to Fight Tuberculosis, AIDS,
and Malaria and the MEASURE Evaluation project.3 Protocol I is intended to assess the underlying
systems and structures supporting the flow of health data through the routine NRHM reporting system,
while Protocol 2 is intended to assess, on a limited scale, if health facilities are collecting and reporting
data in an accurate and timely manner.
2 National Health Systems Resource Centre, HMIS Analysis: Haryana – Palwal – April 2012 – March 2013 (, National Health

Systems Resource Centre, May 2013).

3 Data Quality Tool: Guidelines for Implementation, MS-08-29, 2008.

4

Figure 4: The Five Domains of the Monitoring and Evaluation System
M&E Structures,
Functions and
Capabilities
Indicator
Definitions and
Reporting
Guidelines
Data Collection and
Reporting Forms
and Tools
Data
Management
Processes
Links with
National
Reporting System
Protocol 1 is comprised of a series of questions across five domains that support the M&E system
(Figure 4). The full set of 39 questions can be found in Annex I. Answers to the questions in Protocol 1
across these categories provide a systematic way to catalogue common issues found across multiple
facilities and to compile recommendations for system improvements. Protocol I includes such questions
as: whether or not staff had written copies of data definitions; whether written manuals documenting
how to compile data and how the electronic reporting tools function had been distributed to each and
every facility; and whether the appropriate and routine training had been provided to staff to build their
capacity. The overall objective of this Protocol is to document whether the appropriate system
structures are in place to promote the collection and reporting of high-quality data on a timely basis.
Each question is then scored by the reviewer using a three point rating scale. Full compliance with the
elements of a question elicits a “Yes, completely” response and equates to a 3 score. “Partly” refers to
some criteria for a given measure being met and equates to a 2. “No – not at all” is used when none of
the criteria for a question are met by the health facility, district team or, as applicable, by the state level
NRHM team, and results in a score of 1. Each question can also be marked “N/A” by the reviewer if it is
not applicable, which will keep that specific question from being used to calculate the resulting DQA
scores by category.
Protocol 2 focuses on verifying that the data being captured at the facility level (i.e. on source
documents used by health facility staff and ultimately entered into the DHIS 2.0) are consistently
compiled on an accurate and timely basis. In Haryana, the DHIS 2.0 electronic reporting system allows
each health facility with an internet connected computer and a user identification for the DHIS 2.0 to
directly enter their facilities monthly report. The DHIS 2.0 electronically stores the information by
facility, but has been programmed to compile district level reports by aggregating data from all health
facilities within the district. Thus, for Protocol 2, the DQA teams reviewed reports of the selected
DQA indicators generated by the DHIS 2.0 for each of the facilities and then conducted field visits to
the facilities to review the source documentation for those indicators. In most cases, the field visits
5

required the DQA teams to review two or three distinct paper registers at the facilities to compare the
individual records for the relevant time periods with the compiled reports from the DHIS 2.0.
INDICATOR SELECTION
To initiate work on the DQA, four indicators were selected for review from the 16 being used for
RMNCH+A benchmarking in Haryana. Each of the indicators selected comprises multiple data elements,
which required the review of multiple data sources at each facility. The following are the indicators that
were used in the DQA exercise, with definitions taken from the Health Programme Managers’ Manual
published by NRHM, and the guidelines for where this data should be recorded at each facility4:
 Institutional deliveries to ANC registration
 Data sources: Labour Room Register, Delivery Register, Antenatal Register, Pregnancy Register
 Newborns weighing less than 2.5kg to newborns weighed at birth
 Data sources: Pregnancy Register, Labour Room Register
 Cases of pregnant women with obstetric complications and attended at public facilities to reported
deliveries
 Data sources: Labour Room Register, In-patient Department (IPD) Register, Obstetric IPD
Register, Obstetric Out-patient Department (OPD) Register
 Post-partum sterilization to total female sterilization
 Data sources: Family Planning Register, Operation Theatre Register
According to the official definitions provided for the indicators above, examining the selected data
elements could require the review of no fewer than nine different registers (see Findings section for the
full discussions of issues related to indicator definitions). In reality, none of the health facilities visited by
the DQA team maintained distinct registers as defined above. In most cases, there were two or three
registers to review and often these registers had been drawn by hand in empty log books rather than
being pre-printed with the requisite columns, definitions, and guidelines as defined above. In all site visits
the DQA team found that the data being audited was being recorded on the registers, even when they
were printed by hand.
In addition to the facility reviews detailed in Protocols 1 and 2, a number of system assessment
questions were addressed to the District and State Data Officers in order to obtain multiple
perspectives on the data reporting system. The results from each of the interviews and site visits were
compiled to provide a summary overview of the data reporting processes which make up the findings
for this exercise.
4 Service Provider’s Manual: Understanding Health Management Information Systems, Volume I, National Rural Health Mission,
Ministry of Health and Family Welfare, Government of India, Service Provider’s Manual: Understanding Health Management
Information Systems, Volume I (Nirman Bhavan, New Delhi, January 2011).
6

SITE SELECTION
The Haryana NRHM data management team classified its districts into four quadrants based on their
performance relative to India as whole against the RMNCH+A indicators for the dashboard as shown in
Figure 5.
Figure 5: NRHM Classification of Districts in Haryana
High Performance Districts Promising Districts
Kaithal, Kurukshetra, Mahendragarh/ Narnaul,
Panchkula, Rohtak
Ambala, Jhajjar, Rewari,
Sirsa, Yamunanagar
Low Performance Districts Very Low Performance Districts
Faridabad, Gurgaon, Jind,
Karnal, Sonipat
Bhiwani, Fatehabad, Hisar,
Mewat, Palwal, Panipat
Source: HMIS Data April 2012- March 2013
The DQA team proposed using a purposive selection process that would stratify facilities from across
Haryana based on the varying volumes of services and ensuring that most were relevant to the
indicators selected (i.e. not all intensive services, such as complicated pregnancies, are managed at
lower-level facilities). As it turned out, there were some facilities visited for which the indicators were
not relevant (in the case of ‘post-partum sterilizations’) due to community preferences rather than the
service not being offered. Discussions between the NRHM State Program Managers and the DQA team
led to the selection of three Very Low Performance Districts (Bhiwani, Mewat, and Palwal) and one
High Performance District (Mahendragarh/Narnaul) in order to provide a varied, but not fully random
or representative sample. A total of 10 health facilities from across these four districts were selected to
allow for multiple facility types to be included in the sample. Based on discussions with the State NRHM
team, the accessibility of the facilities within the time available to the DQA team (limited to one week of
field visits) was used to determine the sites to visit.
The Haryana state Data Manager facilitated introductions of the DQA teams to the M&E Officers from
each district, who in turn arranged for the two DQA teams to visit the selected district hospitals and
community health centers (CHCs) during the week of 6 – 11 December 2013.
7

Table 2: Site Visit Schedule
Facility Name
Bhiwani District Hospital
District
Bhiwani
Date of Visit
6 December 2013
CHC Loharu Bhiwani 7 December 2013
CHC Tosham Bhiwani 7 December 2013
Narnaul District Hospital Narnaul 9 December 2013
CHC Kanina Narnaul 9 December 2013
CHC Nangal Choudhary Narnaul 9 December 2013
Mewat District Hospital Mewat 10 December 2013
CHC Ferozpur Zirka Mewat 10 December 2013
CHC Nuh Mewat 10 December 2013
CHC Hathin Palwal 11 December 2013
8

2. DATA QUALITY AUDIT FINDINGS
The Haryana DQA exercise resulted in a relatively uniform set of findings across the sites visited.
Overall, data are being reported on a timely basis with all staff involved in the process aware of the
deadlines and required formats for monthly reporting of data. All facilities use the DHIS 2.0 electronic
web portal to transmit monthly data, submitted by a trained Information Assistant specifically assigned
to each health facility. The DQA team found no significant reports of problems in accessing the DHIS
2.0 or with the computers and IT infrastructure used to conduct the monthly reporting exercises.
As noted in the methodology section, the DQA tools score each facility based on answers to questions
on a 1-3 point scale, with 1 being the lowest. Provided below in the Table 3 is a numerical
representation of the qualitative findings from Protocol 1 (shown in Appendix 1). The table is color-
coded based on the 1-3 ratings to Protocol 1 questions as follows:
Color Code for Protocol 1 ratings
Green 2.5 - 3.0 Yes, completely
Yellow 1.5 - 2.5 Partly
Red 1.0 - 1.5 No - not at all
Overall, Section IV, Data Management Processes, was the lowest scoring component of the review. This
reflects, in large part, the absence of routine data quality reviews, the use of nonstandardized (and in
many cases handwritten) registers and the absence of unique identifiers across health facilities to track
patients and avoid potential double counting. These are structural components of the reporting system
that have a direct impact on the compilation and reporting of data across the system.
Table 3: Assessment of Data Management and Reporting Systems
Facility Name
M&E
Structure,
Functions and
Capabilities
Indicator
Definitions
and Reporting
Guidelines
Data Collection
and Reporting
Forms/Tools
Data
Management
Processes
Links with
National
Reporting
System
Average
(persite)
Bhiwani District
Hospital
3.00 3.00 2.67 1.80 2.50 2.59
CHC Loharu 3.00 2.00 2.33 2.60 2.25 2.44
CHC Tosham 2.67 2.25 2.33 2.00 2.00 2.25
Narnaul District
Hospital
2.33 2.00 2.00 1.60 1.75 1.94
CHC Kanina 3.00 2.00 2.33 1.40 2.00 2.15
CHC Nangal
Choudhary
2.67 2.00 2.33 2.25 2.00 2.25
Mewat District
Hospital
2.67 2.00 2.00 2.00 2.25 2.18
CHC Ferozpur Zirka 2.67 1.50 2.33 1.80 1.75 2.01
CHC Nuh 3.00 2.00 2.33 1.80 1.75 2.18
CHC Hathin 2.67 3.00 2.67 2.00 2.50 2.57
Average (per
functional area)
2.74 2.13 2.36 1.90 2.10 2.25
9

The highest-scoring component overall was Section I, M&E Structure, Functions, and Capabilities, which
is largely due to the use of the DHIS 2.0 at all facilities and districts, the ability of staff to easily navigate
the system, and the consistent availability of access to the DHIS 2.0 via computers and Internet
connections. The DQA team also found that the DHIS 2.0 facilitates the compilation of data across the
NRHM system as a number of the 16 indicators used in the RMNCHA+A dashboard are automatically
calculated rather than requiring the health facility staff to calculate them on their own. For example, the
“Number of cases of pregnant women with obstetric complications and attended at public facilities” is
automatically calculated from the sub-data elements comprising “obstetric complications: those treated
with antibiotics, those treated for eclampsia, those treated with antihypertensive injections, those
receiving blood transfusions after having severe anaemia (Hb < 7 mg),” and those “receiving 100 IFA
tablets to treat iron deficiencies.” This automatic aggregation simplifies the reporting process and
minimizes potential aggregation errors.
Based on the implementation of Protocol 1, the following are the findings, by domain:
 District hospitals and CHCs are well supported by trained Information Assistants. The Information
Assistants were clear about the timeframes, tools, and process for compiling monthly data and
entering that data into the DHIS 2.0.
 Duties for data management, reporting, and review of data prior to submission are clearly assigned.
While none of the health facilities visited had written job descriptions, there was no confusion as to
who was responsible for data capture, review, and reporting.
 At all of the facilities visited, the computer and Internet connections were found to be in working
order with no significant reports of system outages.
 While there is regular reporting of information up from the facilities, there is a clear lack of routine
feedback to facilities on the data that they have reported.
 In line with the prior finding, there is also minimal use of gathered information at the facility level for
reviewing such targets as population coverage for immunizations or outreach for various services.
These two findings appear to be a function of district-facility interactions, rather than of DHIS 2.0
functionality, as the monthly reports can be directly accessed by facility level staff if they so choose.
 Submission dates and processes were found to be well known and followed; written reporting
guidelines, however, are not widely distributed. There were several reports of the guidelines having
been distributed at some point, but none of the facilities visited could provide the DQA team with a
copy of these guidelines.
 There were found to be varying interpretations of indicator definitions among the reporting staff at
the sites visited. For example, for the indicator “Newborns Weighing less than 2.5 kg to Newborns
Weighed at Birth” staff members at some facilities were including newborn weights of 2.5 kg exactly
within the numerator, while the definition clearly states only those weighing less than 2.5 kg.
 In another example, the “obstetric complications” indicator as defined did not directly correspond
to the reporting formats on the monthly HMIS forms. The definition focuses on the diagnosis and
provides examples of symptoms that qualify as complications: eclampsia, obstructed labor and acute
anemia are examples. The monthly HMIS reporting forms, however, use the aggregation of
treatments for complications: provision of hypertensive pills, distributing IFA tablets and conducting
blood transfusions. The DQA exercise team further noted confusion among a number of facility staff
10

surrounding the units of measure to be reported. Whereas “obstetric complications” calls for the
number of women treated to counted, some facilities were found to be counting, for example, the
number of pills distributed to the women instead. This would essentially invalidate the data being
reported and should be corrected as soon as possible.
 The DHIS 2.0 and the monthly HMIS forms used to compile data for entry into the DHIS 2.0 were
found to be consistently used across all the facilities visited.
 However, there was no uniformity found among the source registers used at the service delivery
sites visited. Only one of the sites visited was found to have pre-printed forms to use for data
capture of monthly pregnancies and deliveries. All other sites were using either blank registers
modified by hand to record information, or existing registers created for another purpose (e.g.
Medical Store Supplies) that were modified by adding columns and rows as needed. With the hand
printed registers, the relevant data was consistently available for review by the audit team.
 In line with the prior finding, recording practices on source documents varies widely across the sites
visited. For example, some weights of newborns were recorded with two decimal places, while
others used only one (i.e., 2.1 versus 2.15). This may be due to the lack of standardized forms
(registers) in use and the absence of written indicator guidelines distributed across facilities.
 The staff at the facilities generally demonstrated the capacity to compile data, as required. There
were, however, some cases noted of miscounting due in part to the non-standardized registers and
inconsistencies with alignment of reporting periods.
 Data confidentiality appears to be effectively maintained; data registers are uniformly maintained in a
locked room after clinical hours. However, a number of staff noted concerns with not having a
defined location for the routine storage of patient records.
 There are no unique facility identifiers in use with the reporting system. A cursory review of the
DHIS 2.0 by the DQA auditors did not reveal any obvious duplication of facility names. Given that
there are more than 24,000 primary health clinics in India5, however, there exists a possibility of
duplicating names within the reporting systems at some point, particularly as they either come on
line or are removed from the health system.
 There is a mixed finding with regard to linkages to the national reporting system. There is a clear
and consistent link to national reporting system via the DHIS 2.0 with regard to the data elements
captured and the timelines for reporting. However, there is no system interoperability between the
DHIS 2.0 and the national HMIS web portal, which leads to a complex process of data exports from
the DHIS 2.0 at the state level in order for the national HMIS reporting to be completed.
5 All India Health Status Report, as of 31 December 2013, NRHM website: http://nrhm.gov.in/images/pdf/mis-report/Dec-
2013/1-NRHM.pdf
11

 In addition, there is significant duplicative data capture for the DHIS 2.0 and the MCTS, which is
burdensome for the staff at the health facility level. While the MCTS appears to capture a significant
portion of the data that is also reported through the DHIS 2.0 (not verified by the DQA team during
this exercise, however), there is no electronic interface between the two systems. Streamlining the
MCTS, the DHIS 2.0, and the national HMIS data capture and transfers would significantly reduce
the duplication of effort across the health system.
The following findings are a result of the implementation of Protocol 2 during the DQA exercise, which
entails validation of reported data versus re-counted data and reviewing the timeliness of data reporting.
The latter component could not be reviewed during the DQA exercise as no logs from the DHIS 2.0
were available to the audit team to review reporting timeliness.
Source Documents
 During the site visits by the DQA team, most source documents for the indicators being reviewed
were readily available at the facilities for the audit period in question, in spite of the fact that the
facilities were not using standardized, pre-printed registers.
 Although they were not uniform in their structures, the registers reviewed were generally complete
and well-kept by the staff at the facilities. While reviewing the handwritten documents, the DQA
team had only moderate trouble interpreting some of the results and in most cases all of the
required data elements for the indicators under review were captured.
 As noted above, the lack of standardized source documents with clearly written indicator definitions
has an impact on the understanding of how and what the staff are responsible for counting for each
indicator, which ultimately impacts the data quality.
Indicator Validations
 When reviewing the source documents at facilities and comparing the data counts to the reports
generated by the DHIS 2.0 for the same period, most facility counts were found to match within a
relatively small range of variation (plus/minus 10 percent).
 Those errors that were identified were typically based on varying understandings of the indicator
definitions rather than an inability to accurately tally the totals for data elements. For example, in the
summary table provided below (Table 5) for the validation checks conducted of the indicator on
“obstetric complications” reviewed during the DQA exercise, one CHC was found to be counting
the wrong units (“number of pills dispensed” versus “women experiencing complications”).
 Provided below are the summary tables of the data validations conducted. The table columns show:
the facility name, the verified counts by the auditors, the reported counts from the DHIS 2.0 and a
verification factor that reflects over-reporting (greater than 1.0), under-reporting (less than 1.0) or
an accurate match (exactly 1.0) between counted and reported.
 There were no major validation discrepancies found for the indicator in Table 4, “institutional
deliveries to ANC registrations.”
12

Table 4: Trace and Verify: Institutional Deliveries to ANC Registrations
Facility Name Verified Counts at
Audited Sites
Reported Counts at
Audited Sites
Site Verification
Factor
Bhiwani District Hospital 356 356 1.00
CHC Loharu 10 10 1.00
CHC Tosham 28 28 1.00
Narnaul District Hospital 663 670 0.99
CHC Kanina 0 0 -
CHC Nangal Choudhary 53 53 1.00
Mewat District Hospital 154 154 1.00
CHC Ferozpur Zirka 194 197 0.98
CHC Nuh 180 205 0.88
CHC Hathin 223 208 1.07
 Table 5 below reflects several major data collection errors on validations of “newborns weighing
less than 2.5 kg to total newborns,” each with a unique set of reasons:
 Missing one underweight newborn at CHC Kanina meant a 100 percent reporting discrepancy.
 CHC Nangal Choudry reflects that they were using a reporting period from the 29th of one
month to the 28th of the next, rather than the calendar month. Thus, when compared with the
DQA team’s count of the calendar month, there was a discrepancy.
 CHC Nuh was stopping their counts before the end of the calendar month in order to review
the monthly data internally with their teams before submitting. This resulted in a discrepancy
when compared to the calendar month tally conducted by the DQA team.
 Mewat District Hospital’s discrepancy resulted from a listing in the Admissions-Discharge
register of data ranges for weights (e.g., <1 kg, 2-2.5 kg, 2.5-3.0 kg.) rather than recordings of
the actual weight in kilograms. When staff used the weight ranges, they included those
newborns whose actual weight was exactly 2.5 kg, which, according to the official indicator
definition, should not be included as underweight. This resulted in reporting of the wrong
indicator for this component and should be corrected as soon as possible.
 The reason for the discrepancy noted at CHC Tosham could not be readily identified during the
DQA team’s site visit.
13

Table 5: Trace and Verify: Newborns Weighing Less than 2.5 kg to Total Newborns
Facility Name
Verified Counts at
Audited Sites
Reported Counts
at Audited Sites
Site Verification
Factor
CHC Loharu 2 2 1.00
CHC Kanina 1 0 --
CHC Ferozpur Zirka 23 22 1.05
CHC Nuh 8 19 0.42
CHC Hathin 20 20 1.00
 For the most part, data for the indicator “Cases of Pregnant Women with Complications and
Attended at Public Facilities to Reported Deliveries” showed only minor discrepancies.
 CHC Nuh (noted with a * in Table 6) was found to be counting the wrong units (“number of
pills dispensed” versus “women experiencing complications”) for this indicator. While the Trace
and Verify notes over-reporting of the indicator, in fact, the wrong data was reported.
Table 6: Trace and Verify: Cases of Pregnant Women with Complications and

Attended at Public Facilities to Reported Deliveries

Facility Name
Verified Counts at
Audited Sites
Reported Counts
at Audited Sites
Site Verification
Factor
CHC Loharu 10 10 1.00
CHC Tosham 0 0 --
CHC Kanina 32 32 1.00
CHC Nangal Choudhary 0 0 --
CHC Ferozpur Zirka N/A N/A N/A
CHC Nuh 141 50 2.82*
CHC Hathin 0 0 --
14

 There were no major validation discrepancies found for the indicator “postpartum sterilizations to
total sterilizations”:
Table 7: Trace and Verify: Postpartum Sterilizations to Total Sterilizations
Facility Name
Verified Counts at
Audited Sites
Reported Counts
at Audited Sites
Site Verification
Factor
CHC Loharu 0 0 --
CHC Kanina 10 10 1.00
CHC Ferozpur Zirka 0 0 --
CHC Nuh 0 0 --
CHC Hathin 0 0 --
15

3. RECOMMENDATIONS AND NEXT STEPS

There are a number of clear recommendations emanating from the DQA exercise undertaken by HFG
in December 2013. The key findings described above showed both consistent strengths within the
NRHM reporting system, as well as addressable gaps. The following is a set of recommendations and
next steps suggested for Haryana NRHM to continue strengthening the quality of data reported through
the HIS.
 Develop training plan for districts and health facilities
 As the written guidelines are distributed, a training plan to reinforce the reporting guidelines
should also be developed and implemented. Individuals involved in recording data should receive
an initial training upon assuming their data reporting roles and should also be routinely updated
as indicators, definitions, or timelines are updated.
 Part of the training program for health facility staff should focus on the indicator definitions, how
to properly calculate them and how to interpret them. The Service Provider’s Manual developed
by the central NRHM offices provides useful tools and exercises to promote the understanding
and recording of accurate indicators. However, these manuals are not available to health facility
staff as either reference or training tools.
 Additionally, the trainings should demonstrate to program staff how to use data at the facility
level. For example, using local population data to define target health impacts, creating simple
charts and graphs to track services and targets over time, and comparing data across health
facilities in order to benchmark performance are all effective ways of promoting the use of
information collected at health facilities.
 Provide regular feedback on data submitted into DHIS 2.0
 Establishing routine data review mechanisms, such as a quarterly review of data at the district
level with representatives of local health facilities (e.g. Medical Officers in charge or Information
Assistants), will provide an opportunity to highlight data trends (positive, negative, or potential
anomalies), concerns that may exist with data quality/timeliness, and to provide constructive
feedback to the staff compiling and reporting the data.
 Developing a log system for recording data changed after submission will help document routine
issues that arise and verify the means by which issues have been addressed.
 The Haryana NRHM has recently embarked on an effort to promote the usage of data for
addressing implementation research questions. This effort is believed likely to positively impact
both the reporting of data and the usage side of data for program improvement.
 Implement pop-up windows or “help” section within DHIS 2.0 that would allow users to review
the data definitions and reporting guidelines as they are entering data. It could be implemented
as a full online guide or tagged to specific fields of relevance.
17

 Provide clearly written indicator definitions and reporting guidelines to all facilities
 The written guidelines should include clear due dates for each facility, to whom information
developed by the central NRHM team should be distributed beyond the district levels to ensure
that all health facilities have been trained on them and have them readily available for reference.
 A number of indicator definitions reviewed by the DQA team need clarification. Some are not
well matched to the reporting requirements (e.g. obstetric complications), while others are
being misinterpreted by the reporting staff (e.g. newborns weighing less than 2.5 kg).
Distributing a revised set of written indicator definitions will help to standardize the recording
and reporting of data. The following is a detailed summary of the indicators reviewed during the
DQA exercise that would benefit from refinement and clarification to all staff:
Cases of pregnant women with obstetric complications and attended at public facilities to
reported deliveries
[Cases of pregnant women with obstetric complications and attended]
[Reported deliveries]
18

Concern(s) with current definition:
 The subcomponent “obstetric complications” is vaguely defined. Examples are provided but even
they are sometimes misinterpreted by those recording the indicator. The list should be expanded
and definitions of complications should be more detailed and clear. Reviews of the data registers at
several health facilities indicated inconsistent interpretation of the term “obstetric complications.”
For some examples provided in the manual, such as “eclampsia,” there is no clinical description.
 On the reporting forms, the “Number of eclampsia cases managed during delivery” and “Number
having Hb level <11 (tested cases)”’ are shown in section M1, “Ante Natal Care Services.” However,
“Number of Complicated Pregnancies treated with [IV antibiotics], [IV antihypertensive/magsulph
injection], [IV oxytocin], and/or [blood transfusion]” are in section M5, “Complicated pregnancies.”
This shows that the guidelines reflect the definitional diagnosis while the reporting forms
reflect the possible treatments.
 Some facilities considered “total number of pregnant women given 100 IFA tablets” to be included
in the “Number of Complicated Pregnancies….” indicator. However, rather than reporting the
number of pregnant women treated, they reported the number of pills distributed to
pregnant women, which would clearly result in the reporting of the wrong indicator.
Recommendations:
 Provide a comprehensive checklist of clinical conditions that are classified as meeting the definition
of “obstetric complications” so that labor room teams can actively and rapidly record the data in the
register.
 Consolidate within one section of the reporting forms where the various types of “obstetric
complications” are listed so that tallies are more accurate.
Institutional Deliveries to ANC Registration
[Institutional Deliveries]
19

[ANC Registration]
M1 Antenatal Care Services (ANC) is the health care a woman receives during pregnancy.
ANC starts with ‘history-taking’ and is followed by examination of the woman, which
includes recording weight and height, doing a blood test for anaemia, measuring blood
pressure, and doing a regular abdominal examination as per the guidelines. The woman is
advised for diet, regular antenatal check-ups, and counseled for family planning. She is also
provided with immunisation for TT and IFA tablets along with proper treatment required in
case of any complication.
Ideally, as per the Reproductive and Child Health Programme schedule, the first ANC
check-up is to be done within 12 weeks, preferably as soon as the pregnancy is suspected,
the second ANC check-up between 14 and 26 weeks, the third between 28 and 34 weeks,
and the fourth between 36 and 40 weeks, but due to unawareness, mobility, distance, and
so forth, the timing for the check-ups may vary.
 One challenge with this indicator is that deliveries take place at higher level care facilities with
obstetrical beds, while ANC registrations take place at a much broader range of facilities, including
those without obstetrical services. Because of this, there may or may not be a correlation between
the number of women registered for ANC services and the number of deliveries conducted at any
given facility.
 The ANC registration definition indicates that only new pregnant women registering for ANC
services that month should be included in the count. However, most of the registers reviewed
during the DQA exercise used unofficial or adapted registers made by facility staff that did not
distinguish between first, second, or subsequent ANC visits. For the DHIS 2.0 and HMIS web portal
reporting, the staff members use the registers for data consolidation, not the MCTS, which creates a
tracking sheet for the ANC visits.
Recommendation:
 Use only standardized, NRHM-issued registers entitled “First ANC visit/registration.”
Newborns Weighing less than 2.5 kg to Newborns Weighed at Birth
[Newborns Weighing less than 2.5kg] / [Newborns Weighed at Birth]
20

 The definition does not state explicitly whether newborns weighing exactly 2,500 grams (2.5 kg)
should be included in the list of “less than 2,500 grams.” At some of the facilities visited, newborns
weighing exactly 2,500 grams were included while at other facilities they were not.
Recommendation:
 Add one line of text to the definition to state: ‘Newborns weighing exactly 2,500 grams’ at birth
should not be included in the totals for ‘less than 2,500 grams.’
 Provide proper reporting tools to all district offices and facilities
 In accordance with the written guidelines, each facility should be given an adequate number of
the paper registers it needs to report on each service it provides. A comprehensive maternal
and child health register has been developed and distributed to the state-level teams, but these
have not been uniformly distributed across districts and health facilities.
 Incorporate routine data quality audits as continuous, internal review mechanisms for
facilities
 An ongoing, NRHM-driven process to review source-level data across facilities would assist in
identifying and reducing errors in a timely manner. A simple Excel-based Routine Data Quality
Assessment (RDQA) Tool can be used to document structures in place at facilities, review key
indicators for accuracy and develop action plans for improvement where necessary.
 The RDQA Tool (or similar approach) can be incorporated into the routine supportive
supervision visit carried out by district-level staff to facilities within their district. This would
allow the NRHM to leverage existing systems and relationships while minimizing the need to
mobilize additional resources.
 Streamline reporting systems
 While the electronic data systems used in Haryana eventually feed into the national reporting
system, there are many layers of duplication. The NRHM programs would benefit greatly from
the creation of electronic data interfaces between systems (in the short term) and the
21

consolidation of many of the duplicate systems into a single electronic system with patient-level
records (in the long term).
Source Documents
 Guidelines on maintain source documents
 In all cases during the DQA site visits, the requested source documents reflecting the original
data for the period under review were located and available to the team. However, there do
not appear to be any uniformly followed standards for length of time that source documents are
maintained. NRHM should provide guidelines on timelines for maintaining such records.
Indicator Validations
The Recommendations provided earlier in this section are likely to address the validation issues raised
during the DQA exercise.
Conclusion
Based on the findings from the DQA exercise conducted by HFG, implementing the recommendations
provided here can systematically improve the quality of data being reported and promote the use of that
information for program planning and M&E for the RMNCH+A strategy. Each activity within the
recommendation that is implemented will move NRHM a step closer to helping the RMNCH+A
approach reach its goal of improving maternal and child health outcomes across Haryana.
22

ANNEX 1: DQA PROTOCOL 1 QUESTIONS

23

M&EUnit
AggregationLevels
ServicePoints
1
There is a documented organizational structure/chart that clearly identifies positions that have data management
responsibilities at the M&E Unit.
P Yes
2 All staff positions dedicated to M&E and data management systems are filled. P -
3 There is a training plan which includes staff involved in data-collection and reporting at all levels in the reporting process. P Yes
4 All relevant staff have received training on the data management processes and tools. P P P -
5
A senior staff member (e.g., the Program Manager) is responsible for reviewing the aggregated numbers prior to the
submission/release of reports from the M&E Unit.
P -
6
There are designated staff responsible for reviewing the quality of data (i.e., accuracy, completeness and timeliness)
received from sub-reporting levels (e.g., regions, districts, service points).
P P -
7
There are designated staff responsible for reviewing aggregated numbers prior to submission to the next level (e.g., to
districts, to regional offices, to the central M&E Unit).
P P -
8 The responsibility for recording the delivery of services on source documents is clearly assigned to the relevant staff. P -
12 … how (e.g., in what specific format) reports are to be submitted. P P P Yes
13 … to whom the reports should be submitted. P P P Yes
14 … when the reports are due. P P P Yes
15 There is a written policy that states for how long source documents and reporting forms need to be retained. P Yes
LIST OF ALL QUESTIONS - For reference only (Protocol 1 - System's Assessment)
Supportingdocumentationrequired?
I - M&E Structure, Functions and Capabilities
Component of the M&E System
Check mark
indicates reporting
system level at
which the question
is asked
II- Indicator Definitions and Reporting Guidelines
24

16
The M&E Unit has identified a standard source document (e.g., medical record, client intake form, register, etc.) to be
used by all service delivery points to record service delivery.
P Yes
17 The M&E Unit has identified standard reporting forms/tools to be used by all reporting levels. P Yes
18 Clear instructions have been provided by the M&E Unit on how to complete the data collection and reporting forms/tools. P P P Yes
19
The source documents and reporting forms/tools specified by the M&E Unit are consistently used by all reporting
levels.
P P -
20
If multiple organizations are implementing activities under the Program/project, they all use the same reporting forms and
report according to the same reporting timelines.
P P P -
21
The data collected by the M&E system has sufficient precision to measure the indicator(s) (i.e., relevant data are
collected by sex, age, etc. if the indicator specifies disaggregation by these characteristics).
P -
22
All source documents and reporting forms relevant for measuring the indicator(s) are available for auditing purposes
(including dated print-outs in case of computerized system).
P P P -
23
The M&E Unit has clearly documented data aggregation, analysis and/or manipulation steps performed at each level of
the reporting system.
P Yes
24
There is a written procedure to address late, incomplete, inaccurate and missing reports; including following-up with sub-
reporting levels on data quality issues.
P P Yes
25
If data discrepancies have been uncovered in reports from sub-reporting levels, the M&E Unit or the Intermediate
Aggregation Levels (e.g., districts or regions) have documented how these inconsistencies have been resolved.
P P -
26
Feedback is systematically provided to all sub-reporting levels on the quality of their reporting (i.e., accuracy,
completeness and timeliness).
P P -
27
There are quality controls in place for when data from paper-based forms are entered into a computer (e.g., double entry,
post-data entry verification, etc).
P P P -
28
For automated (computerized) systems, there is a clearly documented and actively implemented database
administration procedure in place. This includes backup/recovery procedures, security admininstration, and user
administration.
P P P Yes
29 There is a written back-up procedure for when data entry or data processing is computerized. P P P Yes
30
If yes , the latest date of back-up is appropriate given the frequency of update of the computerized system (e.g., back-
ups are weekly or monthly).
P P P -
31 Relevant personal data are maintained according to national or international confidentiality guidelines. P P P -
IV- Data Management Processes
III- Data-collection and Reporting Forms / Tools
25

32
… within each point of service/organization (e.g., a person receiving the same service twice in a reporting period, a
person registered as receiving the same service in two different locations, etc).
P P P -
33
… across service points/organizations (e.g., a person registered as receiving the same service in two different service
points/organizations, etc).
P P P -
34
The reporting system enables the identification and recording of a "drop out", a person "lost to follow-up" and a person
who died.
P P P -
35
The M&E Unit can demonstrate that regular supervisory site visits have taken place and that data quality has been
reviewed.
P Yes
36 When available, the relevant national forms/tools are used for data-collection and reporting. P P P Yes
37 When applicable, data are reported through a single channel of the national information systems. P P P -
38
Reporting deadlines are harmonized with the relevant timelines of the National Program (e.g., cut-off dates for monthly
reporting).
P P P -
39 The service sites are identified using ID numbers that follow a national system. P P P -
IV- Data Management Processes
V- Links with National Reporting System
The reporting system avoids double counting people …
26

Improving Data for Decision-Making: Leveraging Data Quality Audits in Haryana, India

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Improving Data for Decision-Making: Leveraging Data Quality Audits in Haryana, India (20)

More from HFG Project (20)

Recently uploaded (20)

Improving Data for Decision-Making: Leveraging Data Quality Audits in Haryana, India