Data collection and reporting of key performance indicators

DATA COLLECTION IN MEASURING
KEY PERFROMANCE INDICATORS
PREPARED BY MN KIRANMAI,
QPS OFFICER,
QUALITY & PATIENT SAFETY,
KSA.

Clinical audit is a process that has been defined as "a
quality improvement process that seeks to improve
patient care and outcomes through systematic
review of care against explicit criteria and the
implementation of change".
In brief, clinical audit provides a method for
systematically reflecting on and reviewing practice.
CLINICAL AUDIT IN PARALLEL WITH
QUALITY IMPROVEMENT MEASURES

KPIs are important for many reasons and among them, the following
are significant
 Helps to assess and ensure if the health care system practices are
effective ( cost effective) and provide a way to demonstrate
system’s efficiency & development.
 Identifies and promotes good practice and can lead to
improvements in service delivery and outcomes for users.
 Provides opportunities for training and education.
 Helps to ensure better use of resources and therefore, increased
efficiency.
 Can improve working relationships, communication and liaison
between staff, staff and service users, and between agencies.
IMPORTANCE OF KEY
PERFORMANCE INDICATORS

KPIs reflect & demonstrate improvement
of service user outcomes

A useful framework has been provided by Donabedian
(1966) who classified topics under three headings
 Structure: The availability and organization of resources
and personnel.
 Process: The activities undertaken, that is, what is done
with the service’s resources.
 Outcome: The effect of the activities on the ‘health/well-
being’ of the service user, that is, changes for the
individual which can be attributed to the clinical
intervention they received.
Areas for KPI MEASURES

THE KEY TO SUCESSFUL MEASURING OF
PERFORMANCE LIES IN THE EFFECTIVE
METHODS OF DATA COLLECTION, ANALYSIS
AND PROVIDING EVIDENCE BASED
CONCLUSIONS
MEASURING THE PERFORMANCE

The heart of clinical audit is the measurement of care
against standard criteria. The actual care given is
measured against the care that is expected to provide.
Each audit criteria should have “an expected level of
performance” or “target” assigned to it.
MEASURING PERFORMANCE

The concept of measurement is “ SMART”
 S- SPECIFIC
 M- MEASURABLE
 A- ATTAINABLE/ ACHIEVABLE/ AGREED
 R- REALISTIC AND RESOURCED
 T- TIME-BOUND-/ TIME FRAMED
CONCEPT OF MEASUREMENT

 Data collection
 Data analysis
 Drawing conclusions from the analyzed
data
 Data reporting
STEPS IN MEASURING
PERFORMANCE

PLANNING DATA COLLECTION INVOLVES
ESTABLISHING THE FOLLOWING
 USER GROUP TO BE INCLUDED, WITH ANY
EXCEPTIONS NOTED
 THE HEALTH CARE WORKERS INVOLVED IN USER’S
CARE
 THE TIME PERIOD OVER WHICH THE CRITERIA APPLY
DATA COLLECTION

 “100% of drugs in our doctors’ bags should be in-
date.”
 Those who've had an abnormal smear, 100% should
have had action taken.
 "At least 80% of eligible women aged 25-65 should
have had a cervical smear in the last 5 years."
 Bed occupancy rate should be below 85%
EXAMPLES OF CRITERIA

The time frame for measuring and reporting data could
be based on the need of the health care system and it’s
standards, type of accreditation ( JCIA, CBAHI and so
on), requirement by the Ministry of health, requirement
by the Quality and Patient Safety measures etc
TIME FRAME FOR MEASURING
PERFORMANCE

To answer that
 You much first decide what your audit question is
 Then you need to decide what data/variables are needed to scientifically
answer the question
 If that data exist in secondary form, then use them to the extent you
can, keeping in mind limitations.
 But if it does not, and you are able to fund primary collection, then it is
the method of choice.
 The type of data required is dependent on the audit question and
objectives. The aim of data collection is to enable comparison of current
practice against the audit standard; therefore the type of data collected
must facilitate this comparison.
PRIMARY DATA: Data that auditor collects
SECONDARY DATA: Data collected by someone else
DATA COLLECTION CHOICE

 What type of data do I need to collect
(quantitative and/or qualitative)?
 What data items will need to be used to
show whether or not performance levels
have been met for each standard?
 What data sources will be used to find the
data?
 Will a data collection tool need to be
designed?
 Will I need to collect data prospectively
and/or retrospectively?
 What size is the target population and will
I need to take a sample?
 How long will data be collected (manually
and/or electronically)?
 How long will it take to collect the
required amount of data?
 Who will be collecting the data?
 How will I ensure data quality? (
accountability, validity & reliability)
QUESTIONS TO BE ANSWERED FOR
PLANNING DATA COLLECTION

Data items should be:
 Relevant to aim / objectives of audit
 Adequate for the study ( not less, not more)
 Define inclusions and exclusions ( define the
population sample)
 Identify sampling method that is adequate for
reflecting meaningful result
 Define the evidences
 Define the criteria
 Define method of collection for each item
 Comparable against standard/ bench mark
 Avoid/ exclude duplications
DATA ITMES

Fundamentally two types of data
 Quantitative – Numbers, tests, counting, measuring
 Qualitative – Words, images, conversations, observations,
photographs
Clinical audit mainly involves in gathering quantitative data that
reflects the actual number of what is being measured against the
standard or bench mark.
Data collected through quantitative methods are often believed to
yield more objective and accurate information because they were
collected using standardized methods, can be replicated, and,
unlike qualitative data, can be analyzed using sophisticated
statistical techniques
DATA COLLECTION METHODS

 The source of data for an audit should be specified
and agreed by the audit team.
 The source specified should provide the most a
 Relevant, routinely collected raw data from existing
sources should be used for the purposes of the
clinical audit as this avoids duplication of information
and work and allows for repeated data collection and
re-audit with minimum effort.
DATA SOURCES

Collect information from:
 Computer registers ( Medica
Plus)
 Review of contents of
medical records
 Data collection sheets ( raw
data tabulated on excel
sheets or other hard copies)
 Data Reports from clinical
units involved ( bed
management, Operation
room, Nursing services and
significant others)
Data Sources

 Be careful of data collection –
choose a topic which does not entail
too much data collection to the
extent it becomes exhaustive with
subsequent loss of enthusiasm
 Sampling – random or systemic
 Only collect essential information
 Use computers based data
collection forms or Manuel data
 Use other staff & delegate – don’t
do all the work yourself
 Set a deadline
DATA COLLECTION

The following principles should apply when a data collection tool is
being developed:
 The data to be collected should be relevant to the objectives and
criteria for the audit and the
 expected performance levels.
 Acronyms, jargon and technical terms should be avoided.
 Definition of terms used should be included where necessary
(involves
 defining terms in the audit criteria and known exceptions).
 There should be space to record exceptions.
 Questions should be episode-specific i.e. relate to a specific
episode of care.
 Closed questions should be used, these should be clearly worded
and contain no ambiguity i.e. clarify the format for the answer
(for example, date: day/month/yr).
 Limit the use of free text or open questions to clinical audits with
qualitative elements as free text is difficult to code and analysis is
very time consuming.
 Filters should be used in order to make the process of completing
the tool as quick and efficient as possible, for example, ‘if Yes, go
to question four’.
 Data items should be presented in a logical order i.e. the tool
should not require the person collecting or analyzing the data to
skip backwards and forwards.
DATA COLLECTION TOOLS

The data collected should be
reliable and valid
 Validity of data is the degree
to which it measures what it is
supposed to measure.
 Reliability of data is the
extent to which a
measurement gives results
that are consistent.
DATA QUALITY

 Depending on the audit, data
may be collected by more than
one person or different people
may be responsible for
completing different data sets.
 There should be no confusion
over terminology. A definition
should be provided for each
data item so that it is collected
consistently (inter-rater
reliability).
 In addition, everyone involved
in data collection should know
and understand who is
responsible for the various
elements including what, how
and where the data is to be
recorded.
WHO COLLECTS THE DATA

To compare actual practice and performance against
the agreed standards, the clinical audit data must be
collated and analyzed.
The basic aim of data analysis is to convert a collection
of facts (data) into useful information in order to
identify the level of compliance with the agreed
standard.
DATA ANALYSIS

STEPS IN DATA ANALYSIS:
 Collect data
 Transfer data into computer based
summary sheets/ spread sheets/
data base for interpretation
 Code the data items if necessary
like gender
 Arrange data in a logical way for
analysis Eg: smaller number to
bigger number, A-Z, descriptive
statistics like waiting time within
one hour, 2 hours etc
 Check the data for errors and
rectify with the original source
DATA ANALYSIS

Data interpretation:
 The type of data analysis depends
on the type of information
collected. This can range from
simple averages and percentages
to sophisticated statistical
techniques.
 Common interpretations are done
using Count, Mean, median, mode,
percentages, rate and standard
deviation.
 Interpretation is displayed in
graphs and charts depending on
what is measured.
DATA ANALYSIS

In clinical audit, the numerator and denominator
should be clearly defined for each measure.
 Numerator : Total number within the denominator
group that met the criterion
 Denominator: Applicable cases for a criterion
DATA ANALYSIS

 Mean is to add all the numbers and divide it by total of the
numbers
Example: (13 + 18 + 13 + 14 + 13 + 16 + 14 + 21 + 13) ÷ 9 = 15
 The "median" is the "middle" value in the list of numbers
Example: Median for 13 + 18 + 13 + 14 + 13 + 16 + 14 + 21 + 13
First rewrite numbers in an order:
13, 13, 13, 13, 14, 14, 16, 18, 21
There are nine numbers in the list, so the middle one will be the (9
+ 1) ÷ 2 = 10 ÷ 2 = 5
 The mode is the number that is repeated more often than any
other, so 13 is the mode.
MEAN, MEDIAN & MODE

STANDARD DEVIATION
 Standard deviation reflects how far we are deviating
from mean

The rates for each indicator are calculated as follows:
 Observed rate = Observed events / Eligible population
 Expected rate = Expected events / Eligible population
 Risk-adjusted rate = (Observed events / Expected events ) * reference population rate
 Smoothed rate = Risk-adjusted rate * weight – reference population rate * (1 − weight)
The counts that are used to calculate the rates of each indicator are determined as
follows:
 Eligible population = for each QI indicator, the total number of a hospital’s discharges
that qualified for the eligible population for that specific indicator
 Observed events = for each QI indicator, the total sum of events that occurred in the
eligible population for that specific indicator
 Expected events = for each QI indicator, the total sum of events expected to occur for
that specific indicator if the hospital had average performance comparable to the
reference population, considering its case mix.
RATE

 Rate is plotted on a run Chart or Control chart
 Percentages are more effective with Pie charts.
 Bar charts or histogram are good for frequency
distribution.
 Pareto Analysis is very similar to Histograms
which Uses the 80/20 rule. Use of percentages
to show importance. Helps to prioritize what to
measure first.
 Scattered diagram is to identify the
correlations that might exist between a quality
characteristic and a factor that might be driving
it ( two variables).
 Fish bone is used to show Graphical
representation of the trail leading to the root
cause of a problem
DATA INTERPRETATION AND REPORTING
USING GRAPHS AND CHARTS

The aim of any presentation of
results should be to maximize
the impact of the clinical audit
on the audience in order to
generate discussion and to
stimulate and support action
planning. Clinical audit
enhances for effective and
logical decision making process
for the governance in the
health care system.
SUMMARY

Data collection and reporting of key performance indicators

Data collection and reporting of key performance indicators

More Related Content

What's hot (20)

Similar to Data collection and reporting of key performance indicators (20)

More from kiran (13)

Recently uploaded (20)

Data collection and reporting of key performance indicators