CCIH-2017-Monitoring-and-Evaluation-Preconference-Module-2

7/12/2017
1
Module 2
Illustrate conceptual models and performance frameworks for
healthcare systems
Anbrasi Edward
Christian Connections for International Health
Pre-conference Workshop July 13, 2017
Linking Mission to Objectives
Develop
Operational
Objectives
Set
Organizational
Goals
Develop
Value based
Strategic
Vision
and
Mission
Craft and
Execute
strategy of
activities
Improve/
Change
Revise as
required
Revise as
required
Improve/
Change
Corrective
action
Step 1 Step 2 Step 3 Step 4 Step 5
Monitor
with Key
Performance
indicators

7/12/2017
2
Road Map
 Formulating the research/program questions, Theory of
change
 Configuring frameworks, determining indicators, data
sources
 Study design, sampling, data collection strategies
 Managing the Evaluation
 Analysis
 Sharing, Reporting, Use, Learning, Action Plans
Configuring Performance Frameworks
 Country-demand driven and country led
 Build on existing in-country processes and experience
 Audience: inclusive, involve civil society and other stakeholders
 Align with leading global innovations
 System framework with a value chain and validated indicators
(avoid collinearity )
 Address epidemiological transition
 Measurement within the scope of the health organization
 Maturity and sophistication
 Mandate, skills, resources and independence to succeed
 Measures, methods, frequency, sampling, weighting
 Financing and cost structure

7/12/2017
3
Considerations for Evaluation
 Program context, objectives, intervention strategies
 External vs participatory, quantitative and formative
 Process and/or outcome measures
 Data sources: clinical obs, record review, interviews
 Enumeration, sample size, design effect, power calculations
 Sampling: simple random, systematic, stratified, cluster (multistage),
non-probability, convenience, LQAS
 Clinical, managerial, cost effectiveness measures
 Other considerations
 Primary sampling unit, unit of analysis
 Informed consent procedures, ethical considerations
 Resources: Time, Personnel, Finance
Theory of Change
http://www.mhealthknowledge.org/sites/default/files/MAMA_Global_MEPlan_FINAL_all_0.pdf

7/12/2017
4
Hierarchy of Research Methods?
 Robust experimental designs to weaker qualitative techniques?
 Contextual application; no right and wrong approach,
appropriateness of the strategy, knowledge base, resource
availability, time, environment, level of analysis.
M&E Overview
 Performance indicators
 The logical framework approach
 Formal surveys
 Rapid appraisal methods
 Participatory methods
 Public expenditure tracking surveys
 Cost-benefit and cost-effectiveness analysis
 Impact evaluation
 Not a comprehensive list! Some are complimentary
 Broad and narrow applicability
 Depends on the need of users; the speed for information and cost

7/12/2017
5
Performance Indicators
 Measures of inputs, processes, outputs, outcomes, and impacts for
development projects, programs, or strategies
 Appropriate study design with formal surveys, analysis and reporting,
indicators enable managers to track progress, demonstrate results,
and take corrective action to improve service delivery
 Setting performance targets and assessing progress toward
achieving them
 Provides an early warning system to allow corrective action to be
taken
 Effective means to measure progress toward objectives, facilitates
benchmarking comparisons between different organizational units,
districts, and over time
 Tendency to include too many indicators, or those without adequate
data sources resulting in expensive, impractical, and underutilized
data
Monitoring Indicators
 Selecting indicators, determining baselines, setting targets - fundamental
aspects of monitoring
 Choosing appropriate number of indicators, well-defined and accurate
monitoring can increase data quality used for reporting and decision-making
 Data informs learning agenda, provides evaluation teams with necessary
information to understand what project or activity results have been achieved
 Include priorities and existing efforts of governments, implementing partners,
and other donors, to the extent possible, to align efforts, reduce data
collection and reporting burdens
 Monitoring plans should be reviewed on a regular basis to ensure that
selected indicators continue to be relevant and useful for management
needs – evolving
 Disaggregated data improve understanding of the progress toward
achievements that an indicator captures, by providing details of the
experiences of subsets of beneficiaries

7/12/2017
6
Monitoring Indicators
 To ensure the provision of consistent, timely, and high quality data,
each indicator is required to have an authentic reference
 Includes indicator definition, source of data, frequency of data
collection, and disaggregation etc
 If data are not of good quality, can be misleading and result in wrong
decisions
 Consider; where will the data come from, what level of quality are the
data expected to be, how will data be gathered and stored to protect
integrity, and the privacy of the participants?
 Direct or 3rd party sources (Government ministry, other)?
Data Quality, Storage Security
 Validity; Data clearly and adequately represent the intended result.
 Integrity; Data have safeguards to minimize the risk of transcription
error or data manipulation.
 Precision; Data have sufficient level of detail to permit management
decision making.
 Reliability; Data reflect consistent collection processes and analysis
methods over time.
 Timeliness; Data are available at a useful frequency, are current,
and timely enough to influence management decision making
 Proper data storage and security are critical to protecting data
integrity, optimizing data usability, and safeguarding potentially
sensitive or personally identifiable information.

7/12/2017
7
Global Reference List
 guide monitoring of health results nationally and globally
 reduce excessive and duplicative reporting requirements
 enhance efficiency of data collection investments in countries
 enhance availability and quality of data on results
 improve transparency and accountability
 Disaggregations that include equity stratifiers (e.g. age and sex, geography,
socioeconomic status, place of residence)
 Data sources: civil registration, vital statistics systems, population-based
health surveys, facility-generated data, routine facility information systems,
health facility assessments and surveys, administrative data sources such as
financial and human resources information systems
Results Chain – Refer to HO

7/12/2017
8
Log Frames
 Helps to clarify objectives of any project, program, or policy
 Aids in the identification of the expected causal links—the “program
logic” results chain: inputs, processes, outputs (including coverage or
“reach” across beneficiary groups), outcomes, and impact
 Leads to the identification of performance indicators, risks which
might impede the attainment of the objectives
 Used to engage stakeholders in clarifying and SMARTening
objectives and designing activities
 Management tool to review progress and take corrective action
 Can impede creativity and needs periodic updating to accommodate
HC environment changes
 Example from PEPFAR
Formal Surveys
 used to collect standardized information from a sample of individuals
or households
 Provides baseline data against which the performance of the
strategy, program, or project can be compared
 Comparing different groups at a given point in time, changes over
time in the same group, actual conditions with the targets established
in a program or project design
 Describing conditions in a particular community
 Providing a key input to a formal evaluation of the impact of a
program or project
 Findings from population sample can be applied to the wider target
group or the population as a whole

7/12/2017
9
Household Surveys
Method
Unicef MICS Multi Indicator Cluster Surveys
USAID, DHS Demographic Health Survey
KPC 30 Cluster Surveys Knowledge Practice Coverage
PEPFAR, PMI President’s Initiatives for AIDS, Malaria
WHO Verbal Autopsy Verbal Autopsy for Non Clinicians
CS-KPC Rapid Catch Child Survival Key Performance Indicators
PETS, WB Public Expenditure tracking surveys
LSMS Living Standards Measurement Surveys
Indonesia FLS Indonesia Family Life Surveys
HEIDE Household expenditure and income data for transitional economies
IHSN International household survey network
DDP WB Development data platform
Facility Performance Measures
Method Description
Service Provision Assessment
DHS Measure
National Sample, 3-5y
Service Availability Mapping, WHO Mapping of service and resource availability, National
and district level decision making, Annual
Health Facility Census, JICA Health assets mapping, National, 5y
FASQ, Measure Facility Audit of Service Quality
QIQ, Measure Quick Investigation of Quality
LQAS, (EPI, HIV/AIDS) Lot Quality Assurance Sampling
IMCI- MCE, WHO, Rapid HFA Focus on child health
GFATM, RBM, EPI, RBF etc Global Initiatives
Modules include system readiness, QoC, provider and client interviews, donor assistance, social
marketing, record reviews, costing etc
Technical Areas: MNCH, STI, TB, HIV/AIDS, FP etc
Management, financing, motivation, design, sampling, duration and types of data collection methods vary

7/12/2017
10
Rapid Service Quality Checklist
• Health Facility ____________
• Observer ____________
• Supervisor _____________
• Date _____________
• Did the service provider
1. Use a sterile needle for each injection? Yes No
2. Use a sterile syringe for each injection? Yes No
3. Give the child all the required vaccinations? Yes No
4. Record the vaccination on the child health card? Yes No
• Outreach sessions
• Were vaccines transported in cold boxes with ice packs? Yes No
Rapid Appraisal
 Rapid low-cost ways to obtain feedback of participants and other
stakeholders
 Provides real time information for management decision-making,
especially at the project or program level
 Providing qualitative understanding of complex socioeconomic
changes, highly interactive social situations, or people’s values,
motivations, and reactions
 Providing context and interpretation for quantitative data collected by
more formal methods
 Findings usually relate to specific communities or localities—thus
difficult to generalize from findings

7/12/2017
11
Rapid Appraisal Methods
 Key Informant Interviews: a series of open-ended questions posed to
individuals selected for their knowledge and experience in a topic of
interest. Interviews are qualitative, in-depth, and semi-structured and
use guides with a list topics or questions
 Focus Group Discussions; a facilitated discussion among 8–12
carefully selected participants with similar backgrounds. Participants
might be in the study, program staff, or stakeholders. The facilitator
uses a discussion guide, note-takers record comments and
observations
 Direct Observation; use of a detailed observation form to record what
is seen and heard at a program site.
 Mini-survey; structured questionnaire with a limited number of close-
ended questions that is administered to 50–75 people. Selection of
respondents may be random or ‘purposive’
Participatory Evaluation
 Collaborative problem solving
 Contributes to learning and leads to strengthened program design/
implementation.
 External expert plays a facilitating role with stakeholders
 Share ideas and encourage team to consider options.

7/12/2017
12
Participatory Methods
 Learning about local conditions and local people’s perspectives
and priorities to design more responsive and sustainable
interventions
 Identifying problems and trouble-shooting problems during
implementation
 Evaluating a project, program, or policy – Mixed methods studies
 Providing knowledge and skills to empower poor people
 Stakeholder analysis; develop an understanding of the power
relation ships, influence, and interests of the various people
involved in an activity and to determine who should participate,
and when
Mixed Methods Evaluation
 In addition to determining impact, it is equally important to understand the
implementation process of the program, their successes and challenges to
bring about improvements in quality of the program and learning for program
improvement and expansion
 Therefore, it is critical that the impact evaluation and formative research are
systematically aligned

7/12/2017
13
Cost-Benefit and Cost-Effectiveness Analysis
 Cost-benefit analysis measures both inputs and outputs in monetary
terms. Cost-effectiveness analysis estimates inputs in monetary
terms and outcomes in non-monetary quantitative terms
 Informing decisions about the most efficient allocation of resources
 Identifying projects that offer the highest rate of return on investment
Impact Evaluation
 Systematic identification of the effects; positive/negative, intended/ not, on
individual households, institutions, and the environment caused by a
program or project
 Helps determine equity, effectiveness, coverage, and outcomes
 Range from large scale sample surveys and control groups are compared
before and after, to small-scale rapid assessment
 Measuring outcomes and impacts of an activity and distinguishing these
from the influence of other, external factors.
 Accountability of investments, informing decisions on whether to expand,
modify or eliminate projects, programs or policies
 Drawing lessons for improving the design and management of future
activities
 Comparing the effectiveness of alternative interventions
 Expensive, time consuming

7/12/2017
14
Study design Definition
Randomized control
trial (RCT)
Involves random allocation of the intervention and comparison (e.g. usual care) to different study groups, including
measurement of the outcome before and after the intervention has been made.
Non-RCT
Observations are made before and after the implementation of an intervention, both in a group that receives the
intervention and in a control group that does not.
Case-Control
Compares people with a specific outcome of interest (‘cases’) with people from the same source population but
without that outcome (‘controls’), to examine the association between the outcome and prior exposure (e.g. having
an intervention).
Cohort
Participants who do not have the outcome of interest are allocated (but not randomly) to receive alternative
treatment/therapy/interventions (experimental group) or the standard of care or conventional treatment
(comparison group) or no intervention (control group) and then followed up to determine whether they experience
the outcome of interest.
Cohort
One group (pre- and post-): Single selected group under observation, with a careful measurement being done
before applying the experimental treatment and then measuring after. Two groups (pre- and post-): Group of
people with a common set of characteristics or a set of characteristics that are followed up for a period of time to
determine the incidence of an outcome.
Interrupted time-
series
Uses observations at multiple time points before and after an intervention (the ‘interruption’) in an effort to detect
whether the intervention has had an effect significantly greater than any underlying trend over time.
Hierarchy of Study Designs
Study design Definition
Case or time series
Observations are made on a series of individuals, usually all receiving the same intervention, before and after an
intervention but with no control group.
Cross-sectional
Both exposures and outcomes are measured at a single point in time, and the prevalence of the outcome is compared
among those with and without exposure.
Quantitative, Other
Systematic empirical investigation of social phenomena via statistical, mathematical or numerical data or
computational techniques other than those listed above.
Cost-effectiveness
Cost Effectiveness Analysis (CEA) is a type of economic evaluation that examines both the costs and health
outcomes of alternative intervention strategies.
Mixed Methods
Combines data collection approaches, sometimes both qualitative and quantitative, into the study methodology. Some
studies combine study designs, whereas others may have a single overarching research design, but use mixed
methods for data collection.
Implementation Science
Scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices
into routine practice, and to improve the quality (effectiveness, reliability, safety, appropriateness, equity, efficiency) of
care. It includes the study of factors impacting behavior at the individual (i.e. provider, patient) and organizational
levels.
Case Study
The study of selected current phenomenon over time to provide an in-depth description of essential dimensions and
processes of the phenomenon. Case studies may involve both qualitative and quantitative data and may focus on
single or multiple cases.
Participatory action
research (PAR)
Systematic inquiry, with the collaboration of those affected by the issue being studied, for purposes of education and
taking action or effecting change.
Qualitative
Used to generate an understanding of complex, unquantifiable phenomena, such as people’s experiences or
perceptions.

7/12/2017
15
Models of Impact Evaluation – Refer to HO
Sample Size and Power determinations
 An important consideration for any evaluation study is sample size
 Power analysis should be done in order to estimate the sample size needed
to detect program effects with a reasonable level of certainty
 The power analysis can take account of various kinds of outcome indicators
(such as continuous, binary, and count measures) and the consequent
statistical requirements for rejecting the null hypothesis assuming small,
moderate or large effect sizes, as appropriate
 Standard power calculations can be done using widely available software
 The longitudinal nature of the measures proposed may require large
samples to accommodate more independent and dependent variables, and
to account for anticipated attrition from the study cohort
 In Impact evaluations timing is essential. Many outcomes of interest may
take months to demonstrate change; Maternal Mortality, Illness Incidence etc

7/12/2017
16
Baseline Data
 Collect data on demographic and socioeconomic characteristics of
women/households
 Basic information on health outcomes based on stage of pregnancy,
newborn care or child health
 If opting to use an RCT design, then it is important that the baseline
information is collected prior to randomizing women into either the treatment
or control group
 The purpose of the baseline data collection is twofold
 First, to assess whether or not the women in the treatment and control group are
comparable
 Second, it will provide insight into their health, behavioral patterns, and other
demographic information prior to enrolling
 RCT, minimize selection bias
Data Analysis
 Data may require some processing and analysis to ensure they are accurate
and make sense, but many data require substantial analysis to reach a state
where they are usable and ready to be incorporated into a report
 The kind of analysis necessary depends on the kinds of data that were
collected and how those data are intended to be used
 Qualitative data will undergo content or pattern analyses to see trends
 Quantitative data may be simple analyses to generate sums or averages,
disaggregated, or they require complex approaches such as regression
analyses or multilevel modeling (logistic, Poisson etc)
 Uses dependent variables designed to reflect the appropriate level of
measurement for each outcome of interest; binary events, such as birth in a
clinic, or other events, such as number of antenatal clinic visits and number
of birth complications, may be measured with count variables
 Include data analysis plan, software etc

7/12/2017
17
Presentation and Use of Data
 Data will be presented using a variety of tools including tables, graphs, and
charts as appropriate
 Key findings will be summarized in PowerPoint presentations
 The data can presented at the monthly and quarterly meetings within the
project, and annually with partners
 The primary use of the data will be to inform the program management if
changes are required for program activities
Evaluation Components
 Health System and District team
 Capacity building: training- technical/managerial/HIS
 Utilization patterns, Quality of Care, equity, efficiency
 Supply chain management, facility preparedness
 Community linkages
 Cost recovery
 Scope, cost, data propriety
 PVO
 Capacity building; training, field visits etc
 Communication, reports
 Staff competency, performance, satisfaction; oversight, personnel
development, compensation systems etc
 Financial management
 Organizational learning and leadership

7/12/2017
18
Evaluation Components
 Program Effectiveness
 Achievement of goals and objectives
 Quantitative: MICS/KPC
 Qualitative: KII, FGD etc
 Community capacity/Health System capacity
 sustainability strategy, scale up, gender,country ownership,
‘smart’ integration etc
 Community
 Participation, empowerment, equity
 Functional community structures: CHW, VHC, Health Facility
Councils and other interest groups etc
Performance Assessment Levels
 Household Surveys
 Health Facility Assessments
 Hospital Accreditation and Performance Audits
 Health System Performance

7/12/2017
19
Health System Performance
Assessment Models
WHO Health Systems Monitoring
The World Health Organization (WHO). Health System Metrics. Report of a Technical Meeting. Glion, Switzerland, 28-29 September 2006.

7/12/2017
20
WHO Health Systems Building Blocks
WHO. Everybody's business. Strengthening health systems to improve health outcomes. WHO's Framework for Action. Geneva. 2007
IHP+

7/12/2017
21
USAID’s Health System Assessment: A
How to Manual 2007
Health Indicator Framework: Canada
Canadian Institute for Health Information, Health Indicators, 2005.

7/12/2017
22
Australia’s National Health Performance Framework
National Health Performance Committee (NHPC) (2001), National Health Performance Framework Report, Queensland Health, Brisbane.
Duran, A. - Assessing Performance of the
National Health System in Spain

7/12/2017
23
Frenk J Dimensions of Health System
Reform, Mexico
Health Policy 27 (1994): 19-34
Guisett, A. Portugal
Functions
the system
performs
Health System
Stewardship
Service Delivery
Health
Financing
Resource
Generation
Ultimate
Goals of the
system
Improved
Health (Level
and Equity)
Responsiveness
Social and
Financial Risk
Protection
Improved
Efficiency
Intermediary
objectives of
the system
Improving
Access
Improving
Coverage
Delivering High
Quality, Safe
Health Services
Healthy behaviours
& Health Promotion
Tackling
Broader Health
Determinants
External Context
Demographic Legal and regulatoryEconomic
Epidemiological Technological Socio‐cultural
Political
Source: Adapted from WHR 2000; Atun 2005

7/12/2017
24
Atun et al. - Multimethods Evaluation
Primary Health Care Reforms in Estonia
Atun R, Menabde N, Saluvere K, Jesse M, Habicht J. “Introducing a Complex Health Innovation-Primary Health Care Reforms in Estonia
(multimethods evaluation)”. Health Policy 79 (2006): 79-91.
WHO’s Performance Assessment Tool for Quality
Improvement in Hospitals (PATH)
Veillard J. et al. A performance assessment framework for hospitals: the WHO regional office for Europe PATH project. Int J Qual Health Care. 2005;
17(6):487-496

7/12/2017
25
EFQM - European Foundation for Quality Management -
Fundamental Concepts of Excellence to Health Care
Based on the PATH Framework
World Health Organization (WHO). 2003. Measuring Hospital Performance to Improve the Quality of Care in Europe: A Need for
Clarifying the Concepts and Defining the Main Dimensions. Copenhagen: WHO Regional office for Europe.
Baldrige Award
The Baldrige National Quality Program. Criteria for Performance Excellence 2009-2010. National Institute of Standards and Technology, Gaithersburg,
MD.

7/12/2017
26
Handler et al, Conceptual Framework to Measure
Performance of the Public Health System
Handler, Issel and Turnhock. “A Conceptual Framework to Measure Performance of the Public Health System”. AJPH 2001; 8: 1235-1239.
Kruk et al, Assessing Health System Performance in
Developing Countries
Kruk ME, Freedman LP. Assessing health system performance in developing countries: A review of the literature. Health Policy 85
(2008) 263-276.

7/12/2017
27
International Development Research Center
Organizational Assessment A Framework for Improving Performance Lusthaus C, Adrien M, Anderson G, Carden F, and Montalván GP
Value Based Healthcare - Sisters of Saint Mary Hospital
1st Recipients of the Malcolm Baldridge National Quality Award
SSM Health Care’s Vision Statement:
Through our participation in the healing ministry of
Jesus Christ, communities, especially those that are
economically, physically and socially marginalized,
will experience improved health in mind, body, spirit
and environment within the financial limits of the
system
SSM Health Care’s Mission Statement:
Through our exceptional health care services, we
reveal the healing presence of God.
SSM Health Care’s Values:
In accordance with the philosophy of the Franciscan
Sisters of Mary, we value the sacredness and dignity
of each person. Therefore, we find these five values
consistent with both our heritage and ministerial
priorities:
. Compassion
. Respect
. Excellence
. Stewardship
. Community

7/12/2017
28
Balanced Scorecard
– “A multi-dimensional framework for describing,
implementing and managing strategy at all levels of an
enterprise by linking objectives, initiatives, and measures to
an organization’s strategy.”
Kaplan & Norton, 1996

7/12/2017
29
Distinctive Advantage of ‘Balanced’ Scorecard?
 Complex health sector with varied stakeholders
Pts, community, civil society, providers, HCO, insurers,
financers, policy makers, politicians
 Distortions created by global agendas for vertical programs
 Increasing demand for accountability
 “Strategic performance drivers”
 Efficient management systems
Generating Scorecards
Groene et al” “The balanced scorecard of acute settings : development process, definition of 20 strategic objectives and
implementation. ISQuA, Aug 2009

7/12/2017
30
JHH
Clinical Service Fiscal Infrastructure
Clinical
Effectiveness
Pt
safety
Workplace
Quality
Patient
satisfaction
Financial
Performance
Revenue
Recovery
Access Volume
Clinical
Number of CABG Surgical Site
Infections per 100 Procedures
0.0
5.0
10.0
Actual (%) 9.20
Target (%) 5.00
FY03-Q3
Number of Sentinel Events
0
1
2
3
4
5
Actual 1 0 1 0 1 2 0 0 0 1 0 1 1
Target 0 0 0 0 0 0 0 0 0 0 0 0 0
09/02 10/02 11/02 12/02 01/03 02/03 03/03 04/03 05/03 06/03 07/03 08/03 09/03
Number of Significant
Medication Errors
0
5
10
15
Actual 4 2 3 2 5 6 4 6 4 4 1 11 4
Target 0 0 0 0 0 0 0 0 0 0 0 0 0
09/02 10/02 11/02 12/02 01/03 02/03 03/03 04/03 05/03 06/03 07/03 08/03 09/03
Rate of Compliance with First
Dose Antibiotic
0.0
20.0
40.0
60.0
80.0
100.0
Target (%) 95.0 95.0 95.0
Actual (%) 93.8 58.8 50.0
07/03 08/03 09/03

7/12/2017
31
Outpatient Satisfaction Rate
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Actual 84.7% 84.0% 85.0% 85.0% 84.6% 84.5% 85.2% 85.7% 85.6%
Target 86.3% 85.0% 85.0% 85.0% 85.0%
Wave I Wave II Wave III Wave IV Wave V Wave VI Wave VII Wave VIII Wave IX
Employee Satisfaction Rate
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Actual 59.0% 69.0%
Target 65.0% 65.0%
FY02 FY03
Inpatient Satisfaction Rate
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Actual 80.5% 80.1% 79.8% 80.7% 80.4% 81.1% 80.1% 81.5% 80.8%
Target 80.7% 85.0% 85.0% 85.0% 85.0%
Wave I Wave II Wave III Wave IV Wave V Wave VI Wave VII Wave VIII Wave IX
Employee Turnover
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
Actual 5.04% 4.11% 3.26% 3.20% 3.59%
FY03-Q1 FY03-Q2 FY03-Q3 FY03-Q4 FY04-Q1
Client Satisfaction
ED Walkout Rate
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
Actual 8.4% 6.0% 5.1% 3.8% 4.9% 5.5% 5.3% 5.9% 8.7% 8.2% 8.4% 8.1% 9.8%
Target 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0%
9/02 10/02 11/02 12/02 1/03 2/03 3/03 4/03 5/03 6/03 7/03 8/03 9/03
HAL Line Turnaway Rate
0.0%
5.0%
10.0%
15.0%
Actual 7.3% 5.7% 2.0% 1.0% 3.7% 3.8% 3.3% 3.5% 5.4% 6.5% 5.4% 6.5%
Target 10.0% 10.0% 10.0% 10.0% 10.0% 10.0% 10.0% 10.0% 10.0% 10.0% 10.0% 10.0%
9/02 10/02 11/02 12/02 1/03 2/03 3/03 4/03 5/03 6/03 8/03 9/03
Hours on Red Alert
0.0
50.0
100.0
150.0
200.0
Actual 27.5 35.9 28.8 18.8 164.9 181.0 110.2 1.2 81.6 0.0 71.2 77.8 34.8
Target 54.7 57.1 119.2 46.2 126.9 144.2 79.1 63.2 38.2 44.4 34.1 25.3 49.2
9/02 10/02 11/02 12/02 1/03 2/03 3/03 4/03 5/03 6/03 7/03 8/03 9/03
Hours on Yellow Alert
0.0
100.0
200.0
300.0
400.0
Actual 317.2 275.5 140.1 82.7 149.5 152.5 202.2 174.6 232.4 230.5 255.2 221.7 282.5
Target 250.6 167.0 183.1 187.4 209.0 261.6 246.6 262.2 117.7 131.7 122.4 114.5 138.5
9/02 10/02 11/02 12/02 1/03 2/03 3/03 4/03 5/03 6/03 7/03 8/03 9/03
Total Admissions
(excluding Nursery)
3000
3500
4000
4500
Actual 3,617 4,015 3,563 3,544 3,899 3,239 3,812 3,798 3,900 3,785 3,732 3,607 3,612
Target 3,615 3,850 3,692 3,503 3,965 3,651 3,958 3,882 3,937 3,828 3,813 3,789 3,625
09/02 10/02 11/02 12/02 01/03 02/03 03/03 04/03 05/03 06/03 07/03 08/03 09/03
Total Number of Surgical
Procedures
3000
3500
4000
4500
Actual 3,818 4,225 3,718 3,481 3,950 3,263 3,837 4,086 4,001 4,001 4,069 3,770 3,777
Target 3,813 4,372 4,122 3,842 4,118 3,833 4,079 4,219 4,150 4,027 4,011 4,137 3,746
09/02 10/02 11/02 12/02 01/03 02/03 03/03 04/03 05/03 06/03 07/03 08/03 09/03
Infrastructure

7/12/2017
32
MDG Scorecard
Afghanistan BSC

7/12/2017
33
Accelerated Reconstruction Efforts to Address Disease Burden
Myriad of challenges impeding
service delivery
Ranks among the highest for
maternal and child mortality
Total public health spending
escalated to S$280 million (‘08-’09),
85% financed by external donor
assistance
Service delivery primarily through
contracting mechanisms
Critical shortage of skilled health
providers estimated at 39% (BPHS)
Demand for female providers who
currently constitute <24% of the
workforce
BSC Design - BPHS and EPHS
 Maternal and Newborn Health
 Child Health and Immunization
 Public Nutrition
 Communicable Diseases
 Mental Health
 Disability
 Essential Medicines
 Blood Transfusion Services to
Support BPHS
 Medicine
 Pediatrics
 Surgery
 Obstetrics and Gynecology

7/12/2017
34
NHSPA Study Design
 Stratified random sampling of upto
25 HF in each province
3 District Hospitals
7 Comprehensive Health Centers
15 Basic Health Centers
Case management observations
and exit interviews on a systematic
random sample of 5 patients <5y and
5 patients ≥5y
Interviews with 5 Providers
randomly selected in each facility
Capacity Assessments:
infrastructure, services, equipment,
drugs, guidelines, etc
Study Sample in Provinces Assessed btw 2004
and 2008
Sample 2004 2005 2006 2007 2008
Provinces 33 30 30 30 29
Facilities
551 612 612 615 600
U5 Patient Observations 2787 2827 2912 2982 2962
>5y Patient Observations 2630 2947 2965 3008 2921
Exit Interviews 5053 5716 5857 5895 5796
Provider Interviews 1388 1418 1666 1827 2175
1 Daikundi province was not included in 2004 as it lacked functional facilities, but was included in the later years.
Helmand, Kandahar, Zabul and Uruzgan were excluded after 2004 and Farah was also excluded in 2008.

7/12/2017
35
69
Stakeholder (MOPH, Donor, NGO) Consensus on:
 Purpose of Balanced Scorecard
 Measurement Domains
 Unit of analysis
 Process & frequency of review/decisions
 Principles for benchmarking
 Short-listing indicators based on validity, importance,
reliability
Designing the Scorecard
Afghanistan BSC
Measure
National
Median
Lower
Benchmark
Upper
Benchmark Badakhshan Badghis Baghlan Balkh Bamyan Farah
A. Patients & Community
1 Overall Patient Satisfaction % 83.1 66.4 90.9 86.4 76.9 90.9 84.7 97.9 82.8
2 Patient Perception of Quality Index % 76.0 66.2 83.9 77.6 66.2 82.2 80.0 84.4 69.4
3 Written Shura-e-sehie activities in community % 34.2 18.1 66.5 35.6 0.0 34.2 17.7 34.5 73.2
B. Staff
4 Health Worker Satisfaction Index % 63.5 56.1 67.9 63.5 57.6 67.9 68.3 61.4 54.4
5 Salary payments current % 76.7 52.4 92.0 54.9 91.8 45.8 53.3 91.4 97.7
C. Capacity for Service Provision
6 Equipment Functionality Index* % 65.7 61.3 90.0 69.6 62.2 57.5 67.3 75.8 66.3
7 Drug Availability Index % 71.1 53.3 81.8 52.9 50.1 72.8 56.1 85.6 9.8
8 Family Planning Availability Index % 61.4 43.4 80.3 54.2 57.9 70.4 64.9 82.7 0.0
9 Laboratory Functionality Index (Hospitals & CHCs) % 18.3 5.6 31.7 31.7 3.8 15.2 0.0 37.0 0.0
10 Staffing Index -- Meeting minimum staff guidelines % 39.3 10.1 54.0 38.0 22.4 42.7 45.8 53.0 57.1
11 Provider Knowledge Score % 53.5 44.8 62.3 48.6 41.6 49.3 54.0 69.0 45.5
12 Staff received training in last year % 39.0 30.1 56.3 68.9 50.9 39.0 52.4 35.5 37.2
13 HMIS Use Index % 67.7 49.6 80.7 60.9 62.7 40.0 72.9 67.7 72.4
14 Clinical Guidelines Index % 34.8 22.5 51.0 18.3 25.5 29.9 16.4 41.9 59.5
15 Infrastructure Index % 55.0 49.3 63.2 63.2 49.7 50.0 58.3 57.9 76.7
16 Patient Record Index % 65.6 56.1 92.5 51.5 98.5 80.7 97.3 64.5 97.1
17 Facilities having TB register % 15.8 8.3 26.6 32.5 27.0 16.1 16.4 0.0 4.3
D. Service Provision
18 Patient History and Physical Exam Index % 70.6 55.1 83.5 54.2 71.7 55.1 85.4 83.6 52.0
19 Patient Counseling Index % 29.6 23.3 48.9 23.3 40.4 29.3 55.3 33.2 16.0
20 Proper sharps disposal % 62.2 34.1 85.0 64.4 34.1 76.9 75.1 85.0 67.8
21 Average new outpatient visits per month (BHC>750 visits) % 22.2 6.7 57.1 27.3 10.0 27.3 71.4 22.2 0.0
22 Time spent with patient (> 9 minutes) % 18.0 3.5 31.2 21.0 30.7 1.2 27.3 12.8 18.0
23 BPHS facilities providing antenatal care % 62.0 28.9 82.8 28.9 49.4 49.7 67.2 88.1 82.8
24 Delivery care according to BPHS % 25.4 10.5 39.3 38.0 36.2 10.5 39.3 38.0 57.1
E. Financial Systems
25 Facilities with user fee guidelines % 90.6 80.3 100.0 94.8 95.6 95.9 28.9 86.1 100.0
26 Facilities with exemptions for poor patients % 84.7 64.4 100.0 68.5 54.6 69.3 84.3 95.6 93.9
F. Overall Vision
27 Females as % of new outpatients % 55.2 46.5 59.7 46.9 45.9 56.0 55.1 55.2 59.0
28 Outpatient visit concentration index CI (-1 to 1) -0.010 0.041 -0.055 0.021 0.024 -0.038 0.025 -0.076 -0.036
29 Patient satisfaction concentration index CI (-1 to 1) 0.002 0.020 -0.018 -0.019 0.000 0.003 -0.007 -0.005 0.020
Composite Scores
30 Upper Benchmarks Achieved % 17.2 10.3 30.8 17.2 6.9 6.9 20.7 34.5 31.0
31 Lower Benchmarks Achieved % 82.8 75.9 89.7 86.2 79.3 86.2 86.2 96.6 72.4

7/12/2017
36
Performance by BSC Domains: National Median Score
↑19*** ↑13.7*
↑11*
↑27***
↑29***
* p<0.05 *** p<0.0001
Edward A, Kumar B, Kakar F, Salehi AS, Burnham G, et al. (2011) Configuring Balanced Scorecards for Measuring Health System Performance:
Evidence from 5 Years’ Evaluation in Afghanistan. PLoS Med 8(7): e1001066. doi:10.1371/journal.pmed.1001066
Domain A: Patient and Community

7/12/2017
37
Domain B: Provider Satisfaction
Domain C: Capacity for Service Delivery
0
20
40
60
80
100
2004 2005 2006 2007 2008
Eqipment Functionality
Drug Index
FP Index
Lab Functionality
Infrastructure Index
clinical guidelines

7/12/2017
38
Domain C: Capacity for Service Delivery cont..
Domain D: Service Provision
0
20
40
60
80
100
2004 2005 2006 2007 2008
Pt Screening
Pt Counseling
Sharps disposal
Ave new outpt
visit/m (BHC 750)
Consultation time
(≥ 10m)
ANC services
Delivery Services

7/12/2017
39
Hospital BSC Trends - 2007 to 2009/10
11.1*
5.8
4.8 3
8.6*
6*
7.6*
* p<0.05
Cascading the Scorecard Lower Upper National Median
2004 2005 2006 2007 2008
A. Patients & Community
1 Overall Patient Satisfaction
66.4 90.9 83.1 86.3 86.0 77.7 81.0
2 Patient Perception of Quality Index
66.2 83.9 76.0 76.2 80.3 77.6 77.5
3
Written Shura-e-sehie activities in
community
18.1 66.5 34.2 54.5 66.4 86.0 94.3
B. Staff
4 Health Worker Satisfaction Index
56.1 67.9 63.5 64.1 68.1 69.0 69.1
5 Salary payments current
52.4 92.0 76.7 90.0 81.3 90.7 82.7
C. Capacity for Service Provision
6 Equipment Functionality Index
61.3 90.0 65.7 67.0 78.7 83.8 88.4
7 Drug Availability Index
53.3 81.8 71.1 83.7 85.7 81.0 86.3
8 Family Planning Availability Index
43.4 80.3 61.4 70.0 82.9 93.7 94.9
9
Laboratory Functionality Index
(Hospitals & CHCs)
5.6 31.7 18.3 36.3 43.3 58.5 64.5
1
0
Staffing Index -- Meeting minimum
staff guidelines
10.1 54.0 39.3 58.0 66.9 63.9 72.1
1
1 Provider Knowledge Score
44.8 62.3 53.5 69.0 68.7 68.7
1
2 Staff received training in last year
30.1 56.3 39.0 74.3 68.9 68.5 71.1
1
3 HMIS Index
49.6 80.7 67.7 65.8 74.9 91.5 92.4
1
4 Clinical Guidelines Index
22.5 51.0 34.8 48.9 61.6 78.3 83.9
1
5 Infrastructure Index
49.3 63.2 55.0 44.6 48.7 54.6 54.1
1
6 Patient Record Index
56.1 92.5 65.6 63.2 69.4 70.0 69.9
1
7 Facilities having TB register
8.3 26.6 15.8 20.6 37.4 53.7 62.9
D. Service Provision
National
SC
NGO/Provincial SC
Facility/Team SC
Two-Way Influence
Two-Way Influence
Two-Way Influence
Facility and QI Team SC
Select Indicators
Provincial SC for managers and NGOs
at the provincial level to benchmark
and improve performance
Organizational SC with all domains
and indices for policy and planning at
the national level
Community SC will include
indicators of CHW/Shura
activities, referral and
feedback on quality of
services
Community SC
BSC
CSC

7/12/2017
40
Preparatory Groundwork
Community Gathering
Performance Scorecard Input Tracking Scorecard
•Divide into focus groups
• Information on entitlements/budgets
• Develop input indicators
• Collect evidence on input use
•Record data
•Divide into focus groups
•Develop performance indicators
•Finalize indicators (5‐8 max.)
•Performance scoring by groups
•Verify High/Low Scores
•Record data
Self‐Evaluation
Scorecard
Empowerment
Transparency
Development
Accountability
Improved health care
and utilization
Feedback and Dialogue
Interface Meeting
Trust
Efficiency
Community Scorecards
Edward, A., K. Osei-Bonsu, et al. (2015). "Enhancing governance and health system accountability for people centered
healthcare: an exploratory study of community scorecards in Afghanistan." BMC Health Serv Res 15: 299.
Siadara Facility - CSC
Indicators Round 1 Round 2
(3m)
Round 3
(6m)
Reasons
Provider SC
Water (dysfunctional water pump) 5 10 10 Water pump is repaired
Lack and non-use of IEC materials 8 10 9.5 Additional IEC materials and old copies replaced with new
Damaged clinic roof due to snow
avalanche
9 8 10 Metal roof of clinic was repaired
Inadequate medical equipment 7 6 7 The medical equipment requires replacement, very old
Clinic cleanliness 7 9 9.5 Right now even the clinic guard is aware of preparing 0.5 % chlorine
solution. But incinerator is not available
Waiting time 9 10 10 patient load is low, separate OPD for male and females
Clinic management 9 9.5 10 We work as a team, interpersonal relationship is good, clinic is
managed responsibly
Accurate clinical examination 7 9 9.5 When patient volume is high, the examination time is not satisfactory
especially for ANC, PNC and IMCI
Good behavior among the
personnel
10 10 10 No problem, we work as a team
Community SC
No Water in the clinic 5 10 10 Water pump is repaired and safe water is available for all
Electricity in clinic 3 6 10 Initially no electricity, the Provincial Rehabilitation Team (PRT)
promised to install solar power and eventually did install it.
Medicines 7 10 10 Quality is good, but inadequate quantity. They prescribe ‘white tablets’
(Paracetamol) for all conditions. Now clinic has all medicines required
for each patient for the condition
Clinic staff adequacy 10 10 10 Adequate staff and always present on the job
Waiting room seating 5 10 10 Patients stand due to insufficient chairs. More chairs were provided.
Staff Punctuality 10 10 10 They are always present
Staff behavior with patients 10 10 10 Good behavior of staff, all are satisfied with them
Patient Counseling 10 10 10 When we get treatment, they give good counseling on meds, return
Patients Waiting Time 10 10 10 We do not wait for long time we are satisfied
Clinic cleanliness 10 10 10 Clinic is clean, no flies, mosquitos, and they disposed the garbage

7/12/2017
41
Siadara Action Plan
Indicator Action Proposed Who? Timeline Implementation Research Observations
Water supply Request sent to AADA by Shura following
the monthly meeting with the facility staff.
The clinic in charge coordinated it through
the Yakawlang district governor, Bamyan
PPHD and NGO
Clinic in
charge, AADA,
PPHD, Shura
3m after the
shura
meeting
Multiple negotiation meetings and follow up.
Staff paid for water pump from salary and
eventually reimbursed
IEC materials Clinic in charge requests staff to prepare a
list of IEC materials and forward the request
to NGO
Clinic in
charge, NGO
1m Follow up requests for 2m. but materials
were not received.
Condition of clinic
Building (damaged
roof)
Clinic in charge address the issue with
NGO and PPHD
Clinic in
charge, NGO,
PPHD
1-6m The DHO and District Governor were also
engaged in processing the request
Clinic equipment
and infrastructure
(waiting room chair)
Clinic in charge requests all departments
for equipment needs, and submits requests
to NGO
Clinic in
charge, NGO
3m Discovered chairs in storage and
transferred to waiting area. Though shura
were not indicated in action plan there were
involved in all decisions.
Clinic hygiene Clinic in charge to train staff on infection
prevention (IP), create a plan for the clinic,
and follow up
Clinic in
charge and
staff
Ongoing URC NGO working on QA, set standards for
IP, trained the staff and helped with action
plan for different sections of the clinic. The
staff followed up daily. The staff engaged in
a self-audit, till they achieved 100%
compliance, and were awaiting an external
evaluation by URC
Accuracy of exam ,
pt education and
time spent with pt
Pt triaging to avoid ‘noise’ , time spent with
pt must be considered
staff Ongoing According t BPHS, spend at least 9min with
pts The guard was involved in determining
the reason for clinic visit and guides the pts
to the specific area
BSC in Zambia
Better health Outcome through Mentoring and Assessment (BHOMA)
project in 3 Rural Districts1
 3 Strategy Approach
 District Strategy: QI Teams
 HF Strategy: Establish clinical care standards, self assessment
reports, leadership training
 Community Strategy: 60 CHWs ($60/m), Health Committees,
TBAs
 Linked Facility Surveys: HF and HHS
 7 Domains; Pt and Community, HR, Service capacity, Finance
(action plan), Governance, Service Provision + Overall Vision
Mutale et al; Measuring Health System Strengthening: Application of the BSC approach to rank baseline performance of three rural districts in
Zambia; PLOS; March 2013; Vol 1

7/12/2017
42
PAHO Renewing PHC in the Americas
Multiphase Strategy for Design of a 3-Tier BSC
Phase Phase
1
•JHU: Selected Review of HSPA Models, June 2010
Phase Phase
2
•PAHO HSPA Experts Meeting , June 8‐9: 2010
Phase Phase
3
•JHU/PAHO TAG Review of Expert Recommendations, Prioritize HSPA domains and indicators, July 27‐29, 2010
Phase Phase
4
•JHU Design of PAHO HSPA Conceptual Model and BSC Tool
Phase Phase
5
•JHU and PAHO TAG Review and Development of HSPA Model and 51 Indicator score Card
Phase Phase
6
•PAHO Country Stakeholder Meeting ‐ Cuba, December, 2010
Phase Phase
7
•JHU/PAHO TAG Review of Cuba Meeting Recommendations and Proposed Changes to BSC
Phase Phase
8
•JHU Redesign of Conceptual Model and 118 Indicator BSC
Phase Phase
9
•JHU and PAHO TAG Meeting: Review of Revised HSPA and BSC Levels for Country Implementation, December 14,
2011 – 3‐Tier 110 Multi‐indicator framework
Phase Phase
10
•Country Consultation and Contextualization using Delphi tool and Pilot of PAHO BSC, April 2013
I ‘feared’ when they mentioned
a BSC for HSP
Minister of Health,
El Salvador

7/12/2017
43
Executing the BSC
 5 Principles
 Translating Strategy into operational terms
 Align the organization to the strategy
 Make strategy everyone's everyday job
 Make strategy a continual process and
 Mobilize leadership for change
 3 Functions of the BSC
 Measurement system
 Strategic Management system
 Communication tool
PAHO PHC Oriented BSC

7/12/2017
44
WHO People Centered Healthcare Systems
WHO Championing QI for PHC

7/12/2017
45
World Vision Research Study
 5 year Multi-country Multi Site
Mixed Methods Research Study
on Impact of Integrated Package
of Community Based Interventions
for MNCH
 2012-2017, $3.3M in partnership
with 4 Country Research
Institutions
Sustainable Development Goals & CHNIS

7/12/2017
46
Disease Burden for IMCI Conditions, Immunization, Nutrition
Outcomes
Reported (past 2 weeks) Cambodia
(n=3813) %
Guatemala
(n=4096) %
Kenya
(n=6407) %
Zambia
(n=2648) %
Diarrhea 23.3 17.8 8.0 16.1
Cough or difficult breathing 19.9 15.7 5.4 9.7
Fever 46.8 22.1 15.1 22.2
Full Immunization 56.1 66.5 39.3 63.1
Measles Vaccination 62.9 70.8 54.6 70.8
Underweight (<-2SD) 39.9 26.9 14.8 11.1
Severe Underweight (<-3SD) 19.6 11.5 5.8 2.6
Stunted (<-2SD) 47.5 48.3 40.3 37.4
Severe Stunting (<-3SD) 28.6 20.7 22.8 14.7
Wasted (<-2SD) 6.0 10.6 11.3 3.8
Severe Wasting (<-3SD) 1.4 5.1 5.8 1.3
Anemic (<11g/dl) 74.8 - 57.2 -
Severe Anemia (<7g/dl) 1.5 - 4.0 -
Quality of Screening and Assessment, IMCI
Guatemala Cambodia Kenya Zambia
Variables N % N % N % n %
Clinical Observations of Sick Children 35 39 164 100
Primary complaints
Cough 16 45.7 7 17.9 44 27.3 23 23.0
Difficulty breathing 2 5.7 2 5.1 5 3.1 5 5.0
Diarrhea 4 11.4 3 7.7 19 11.8 39 39.0
Fever 8 22.9 20 51.3 127 78.9 31 31.0
Vomiting 1 2.9 na na 14 8.7 1 1.0
Provider Asked/Checked
Age of child 35 100 39 100 157 95.7 88 88.9
Duration of complaint 34 100 38 97.4 161 98.2 97 97.0
Child weighed 25 71.4 31 79.5 145 88.4 43 43.0
Height measured 21 60.0 1 2.6 28 17.1 16 16.2
Check weight against growth chart 19 55.9 6 15.4 36 22.0 28 28.3
Assessed nutritional status with MUAC 11 32.4 2 5.1 19 11.6 4 4.0
Sought care from CHWs 1 2.8 6 15.3 3 1.8 12 12
Mean consultation time (minutes) 30 12.6 37 10.5 161 8.5 98 8.2

7/12/2017
47
Child Caretaker Counseling
WV Zambia Impact Analysis ScorecardComparison Intervention Difference of Difference
Baseline Final Baseline Final
DOD
Mean
SE p-valueMean/% SE Mean/% SE Mean/% SE Mean/% SE
A. Demographics
Mean family size 4.7 2.0 5.0 1.9 5.3 2.4 5.5 2.5 -0.2 0.1 0.163
Mean assets 2.4 1.7 2.7 1.5 2.3 1.8 2.3 1.6 -0.3 0.1 0.002
Head of HH ever attended school 90.4 0.92 87.2 1.0 93.1 0.8 93.7 0.8 3.9 1.8 0.029
Health insurance2 - - 1.3 0.1 - - 0.6 0.1 0.0 - -
Household severe hunger 1.6 0.4 7.6 2.0 7.2 0.7 18.7 1.9 5.6 2.4 0.020
Access to adequate sanitation1,2,3 80.6 1.1 98.3 0.4 82.3 1.1 94.0 0.7 -6.0 1.7 0.000
Access to clean water1,2,3 10.8 0.9 99.1 0.3 38.6 1.4 93.5 0.7 -33.4 1.8 0.000
World Vision child sponsorship 17.3 0.9 7.6 0.8 5.9 0.6 3.7 0.6 7.5 1.5 0.000
B.1. Child Health and Nutrition
Prevalence of ARI in past 2 weeks 4.6 0.6 4.1 0.6 14.8 1.0 4.9 0.7 -9.4 1.5 0.000
Prevalence of fever in past 2 weeks 11.2 0.9 19.4 1.2 33.2 1.3 16.6 1.2 -24.9 2.3 0.000
Prevalence of diarrhea in past 2 weeks 14.5 1.0 16.8 1.1 17.7 1.0 13.0 1.0 -7.1 2.1 0.001
Care seeking for ARI 1,3 25.0 12.5 33.3 7.9 33.7 4.6 84.6 5.8 42.6 17.4 0.015
Care seeking for fever 26.5 6.3 58.5 3.9 25.7 2.8 86.2 3.2 28.5 8.7 0.001
Care seeking for diarrhea 73.5 3.2 41.5 3.6 65.4 3.1 51.1 4.3 17.7 7.2 0.013
Child use of ITN in the previous night1,3 54.0 1.4 78.2 1.3 55.9 1.4 42.7 1.5 -37.5 2.8 0.000
Exclusive breastfeeding <6 months3 100.0 0.00 66.2 3.8 100.0 0.0 67.9 3.6 1.6 5.7 0.778
Breastmilk and complementary feeding 6-9 months 6.1 2.9 69.4 5.9 4.7 2.6 53.4 5.9 -14.6 9.4 0.122
Antibiotic treatment of ARI3 62.3 6.2 74.4 6.7 41.1 3.5 28.0 6.4 -25.2 12.2 0.040
Antimalarial treatment of fever 32.5 4.2 11.5 2.2 25.9 2.5 24.9 3.3 20.0 6.1 0.001
Children with diarrhea given ORT1,3 100.0 0.0 82.2 3.0 100.0 0.0 86.5 3.0 4.3 3.6 0.231
Underweight prevalence1 10.1 0.8 10.8 1.0 12.0 0.8 11.7 1.0 -1.0 1.8 0.595
Stunted prevalence1,2, 3 39.1 1.3 34.5 1.6 35.9 1.2 34.7 1.6 3.3 2.9 0.247
Wasted prevalence1,2, 3 3.6 0.5 4.5 0.6 3.9 0.5 3.5 0.6 -1.4 1.1 0.224
Severe acute malnutrition 1.4 0.3 1.6 0.4 1.3 0.3 1.7 0.4 0.1 0.7 0.851
Full vaccination3a 69.4 2.8 69.6 3.1 56.7 3.1 56.3 3.0 -0.6 6.0 0.920
Measles vaccination1 91.4 1.7 99.5 0.5 87.1 2.1 98.0 0.9 2.8 3.1 0.231
Received first dose vitamin A b 69.8 2.8 97.0 1.1 67.3 2.9 88.5 2.0 -6.0 4.8 0.210
DPT vaccination (3 doses) 1,3 78.0 2.5 91.5 1.9 68.1 2.9 90.9 1.9 9.3 4.9 0.056
B.2. Maternal and Neonatal Health
Antenatal care (ANC) index3 3.3 0.9 3.3 1.0 3.0 1.0 3.9 0.7 0.8 0.1 0.000
Newborn care index3 2.7 0.5 2.8 0.5 2.7 0.5 2.9 0.3 0.1 0.0 0.117
Contraceptive prevalence rate1,3 62.7 1.7 70.5 2.7 59.3 1.9 59.9 2.7 -7.2 4.7 0.124
Mean month of first ANC visit 4.8 8.0 3.8 1.0 4.7 5.9 4.2 1.2 0.5 0.5 0.257
At least 1 ANC visit1,3 95.5 0.8 78.4 1.8 98.7 0.5 97.4 0.6 15.8 2.0 0.000
4 or more ANC visits1, 3 48.4 2.0 76.0 2.1 53.8 2.1 60.6 2.0 -20.8 4.2 0.000
Skilled provider for ANC 94.7 0.9 97.6 0.7 96.7 0.8 97.1 0.7 -2.6 1.5 0.091
Facility based birth 21.4 1.7 65.3 2.0 24.7 1.9 83.4 1.5 14.8 3.5 0.000
Skilled attendant at birth1,2, 3 82.8 1.6 95.5 0.9 79.0 1.8 94.3 0.9 2.5 2.6 0.338
PNC less than 1 day after delivery1,3 10.4 1.4 15.1 1.7 7.3 1.2 58.3 2.0 46.3 3.5 0.000
Skilled provider for PNC 98.5 0.6 99.1 0.4 97.5 0.8 99.0 0.4 0.9 1.1 0.414
C. Quality of Care
Children U5 mean utilization (previous month) 231.2 162.4 198.2 266.9 239.6 138.3 259.4 246.0 52.8 136.4 0.701
Mean supervision visits in last 6 months4 3.3 2.3 7.0 5.4 2.2 1.5 11.0 9.8 5.1 3.8 0.186
Service provision score3 6.4 0.7 5.2 0.4 6.4 0.5 5.3 0.7 0.2 0.4 0.681
Essential medicines score3,4 5.8 0.4 6.0 0.0 5.9 0.3 6.0 0.0 -0.1 0.2 0.447
Emergency obstetric care services score1,3 3.4 1.6 4.9 0.3 4.3 0.9 5.0 0.0 -0.7 0.6 0.231
Children U5 Patient assessment score 3.6 1.5 4.4 2.5 6.0 1.6 3.6 2.2 -3.3 0.6 0.000
Children U5 Patient counseling score 1.8 1.4 4.5 1.2 3.2 1.7 4.2 1.8 -1.7 0.4 0.000
Children U5 Caretaker satisfaction score4 6.6 2.0 6.5 1.0 6.3 1.4 7.0 1.1 0.9 0.4 0.037

7/12/2017
48
Performance Metrics
Lord Kelvin
“You cannot manage what you cannot measure”
Albert Einstein
“Not everything that can be counted counts and not
everything that counts can be counted”
Future considerations…

7/12/2017
49
Church as an integral part of the
healthcare system
ONS Happiness Indices
 Overall, how satisfied are you with your life nowadays?
 Overall, to what extent do you feel the things you do in
your life are worthwhile?
 Overall, how happy did you feel yesterday?
 Overall, how anxious did you feel yesterday?
 All answered using a 0 to 10 scale where 0 is 'not at all'
and 10 is 'completely'.

7/12/2017
50
Re‐imagining the 21st Century Physician: IHI
Health care is locked in a 100-year-old paradigm. We gained over
30 years of life expectancy in the 20th century, but only five of
those years are directly attributable to medical care. So how can
we break out of this costly, reactive, biomedical, and disease-
oriented approach ---- and focus instead on the key factors that
contribute to good health? One answer is to change the way
doctors are trained
IHI Open School
What does the new physician look like?
What new competencies do medical professionals need?
And what disruptive innovations can help our current health care
system adapt to a new paradigm?

CCIH-2017-Monitoring-and-Evaluation-Preconference-Module-2

More Related Content

What's hot (20)

Similar to CCIH-2017-Monitoring-and-Evaluation-Preconference-Module-2 (20)

More from Christian Connections for International Health (20)

Recently uploaded (20)

CCIH-2017-Monitoring-and-Evaluation-Preconference-Module-2