Developing and Implementing a Patient Reported Experience Measure

Transforming Participation in CKD
Developing and Implementing a
Patient Reported Experience
Measure
Rachel Gair
Person Centred Care Facilitator

The Passive Patient
Chronic Kidney Disease National Programme | Rachel Gair | 2
• Healthcare can be
profoundly disempowering
• But most patients want to be
treated as active participants
– as co-producers of health.

To a person centred approach…
| 3Chronic Kidney Disease National Programme | Rachel Gair

Background
NHS health policy:
emphasises the crucial role of patient experience as a core component of
high quality care, alongside effectiveness and safety.
Domains of quality
Effectiveness of care (PROMs) Does it reduce symptoms, improve function,
improve quality of life?
Safety (PROMs) Does it cause harm eg mortality, complications?
Experience of care (PREMs) What do patients think of the process of care eg
dignity, information, trust in staff, cleanliness, timeliness?
Hutchings et al Improving health world wide – Kings Fund

Why Collect PREM?
Measuring patient experience is important:
• To guide service improvement
• People’s experiences of care may be linked to clinical outcomes and
costs
>Improving patient outcomes can increase patient experience ratings by 10%
>Improving patient experience ratings will cause a 3% improvement on outcome
scores.
Doyle C et al systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 2013; 3(1): .

Gaps
Despite a wealth of initiatives:
• The National Patient Survey programme
• NHS Patient Experience Framework
• NICE quality standard for patient experience in adult NHS services
• The collection of patient-reported experience measures (PREMs) and
• Patient-reported outcomes (PROMs) –
There are still gaps in understanding at a local and national level about how
healthcare is really experienced

What is PREM?
Experience – is subdivided
• Satisfaction measures - which allow patients (or their carers)to report their
subjective view, or perception of, the treatment received (e.g. would you
recommend the service); and more recently
• Patient Reported Experience Measures (PREMs) which collate patients’
objective experience of care. By focusing on specific aspects of the process of
care (e.g. were you seen on time)
PREMs seek to remove the subjectivity associated with measures of satisfaction.

What is PREM?
• Patient reported experience measures (PREMS) are used to understand
patients’ views on their experience while receiving care, rather than the
outcome of that care.
• Using information on both patient experience and outcomes enables us to
have a broader understanding of service quality from patients’ viewpoint.
• It may be possible to have a service which provides good outcomes but a poor
experience, or a good experience but poor outcomes.

Challenges
Even when health professionals express positive opinions about using patient
feedback, this does not necessarily translate into changing practice. Side-
lined as too subjective and divorced from the ‘real’ clinical work of measuring
effectiveness and safety.
So what are best ways to:
• assess patient experience
• act on patient feedback, especially as patients move between different parts
of the health service.
• Ultimately translating into redesign and delivery of healthcare services

What are the questions the TP – CKD programme is asking?
• Can PAM/CS-PAM/PROM/PREM measures be collected routinely
within renal units?
• Is the PAM related to PROM/PREM/Clinical Measure results?
• Can we introduce interventions that will increase a patient’s and
teams activation?
Co-production as a core value

TP-CKD – Measurement Workstream
• PREM tool developed with members of the Measurement Workstream of
the Think Kidneys Programme – TP-CKD.
• Had input from kidney patient charities such as the National Kidney
Federation, BKPA and the UKRR Patients’ Council.
• Identified available instruments to measure patient-reported experience:
• Exploratory literature review
• PREM tool comprises 43 survey questions – pick and mix!
• Covers 12 themes – access to renal team – transport – communication –
Shared Decisions

Overview of identified instruments (1)
Specific for renal services
1.CHOICE Satisfaction questionnaire (Dialysis)
www.ncbi.nlm.nih.gov/pubmed/24840250
2.CAPHS in-center hemodialysis survey (in-centre HD)
https://cahps.ahrq.gov
3.Scottish Renal Patient Experience survey (RRT)
www.healthcareimprovementscotland.org/previous_resources/
performance_review/renal_patient_survey.aspx
4.CQ Index for dialysis (Dialysis)
5.Satisfaction of patients on chronic HD and PD (Dialysis)
6.Consumer Satisfaction Questionnaire (PD)

Overview of identified instruments (2)
Specific for renal services – developed locally
7.Bradford Teaching Hospitals (Tx)
8.King’s Fund Patient experience survey (Dialysis)
Specific for renal treatment
9.Renal Treatment Satisfaction Questionnaire (RRT)
10.Good Dialysis Index (HD)
www.nocturnaldialysis.org/good_dialysis_index.htm
Non-disease specific
11.National NHS Inpatient survey (hospital care)
www.cqc.org.uk/content/inpatient-survey-2014
12.WHO Responsiveness scale (general healthcare)
www.who.int/responsiveness/papers/paper21.pdf?ua=1

Summary points
No instrument available to measure patient experience across all RRT
modalities developed specifically for CKD patients not on RRT.
Most validated instruments had been validated in one particular
healthcare context.
Some aspects of patient experience appeared in the majority of
instruments:
Relation with the nephrologist Relation with other dialysis staff
Information provision and decision making Physical Environment

Developing and Implementing a Patient Reported Experience Measure

PREM Strategy
The UK Renal Registry (UKRR) and British Kidney Patient Association (BKPA) have
developed a national strategy for an annual PREM survey.
The PREM is a survey for all renal units and their satellites to take part in.
The national PREM strategy and data collection aims to:
1. Facilitate benchmarking of kidney patient experience
2. Inform service improvement for individual renal teams

Process
This is a ‘Pilot’ – started with the 52 units in England
Discussed at CD meeting – an appetite for a national CKD PREM
Letters to CD - inviting participation + outlining implementation
PREM tool: 43 questions and currently a hard copy survey for all renal patients,
regardless of the type of treatment or where they are receiving it
Suggested 2 week window for PREM tool to be handed out
Renal Unit Guidance – posters + leaflet + 650 surveys to each unit + SAE + box for
returns
Returns via courier in box to UKRR

Implementation – what we have learned
• We were unprepared for appetite + enthusiasm!
• A very tight turn around – units needed more preparation time
• Often didn’t read RUG – preferred to call and discuss
• Needed to extend time for returns by 3 weeks
• Wanted to be told what to do rather than look at local needs
• Don’t have an understanding of % of returns against those handed out
• Amount of SAE required – Tx patients
• Electronic options - PV

Newcastle’s experience
• Implementation working group – clinical + patient lead
• Patient volunteers from KPA
• Senior nurses in each area committed to delivery - responsibility
• Whole team engagement – DM excited about returns – housekeeper helped
– led by HCA on ward
• Understanding of surveys handed out and return rate – followed up ( too
many surveys, don’t hear anything, language barrier)
• In the main patient’s happy to participate

What next - Validation
This is being conducted by University of Hertfordshire
Dr David Wellsted + Dr Janine Hawkins – the study team
Validation working group – Ken Farrington, Sabine Van Der Veer, Paul Bristow,
Claire Corps, Karen Thomas, Rachel Gair, Annie Taylor
HRA have evaluated this as service evaluation (not research) so HRA approval /
REC not required
Ethical approval sought from University of Hertfordshire Health & Human
Sciences Ethical Committee

Validation Process
Aim of PREM: Accurately & reliably measure the experience people with CKD
have of care they receive from RU
Aim of Validation: Does the PREM measure what we want it to measure?
Two key aspects of validity are already established:
Face validity (the PREM looks like it is asking the right things of the right
people in the right way)
Content Validity (it includes what experts would expect it to)

Validation – What does it entail
PREM Validation Project – two key areas of robustness
Validity (is the PREM measuring what we think it is?)
Reliability (are patient responses consistent across the different themes in
the PREM, and over time?)
How? Three phases of work…
1. Pilot data analysis
2. Test retest exercise
3. Cognitive interviews

(1) Data Analysis
The team at UH will analyse the pilot dataset (ie, 8,000+ responses from the
PREM this summer), including:
1. Construct validity (factor analysis) – what does the clustering of items
(sub scales) tell us about the scale (eg are clusters consistent with face
validity)
2. Internal reliability (Chronbach’s Alpha) – which items are consistent with
overall experience (weaker items can be removed)
3. External Indicators – Eg does PREM unit data align with the Friends and
Family data?
4. Normative modelling – assessing a person’s score against a population
standard appropriate for that person

Test-retest reliability
Scale stability - Does the PREM measure patient experience consistently,
regardless of when a patient answers it?
• Sub sample of 5 renal units in Wales – aiming for 200 responses
• Patients complete the PREM twice in a short space of time (2nd
weeks of
November and December)
• December PREM includes a few questions about any changes in their
life / treatment / circumstances
• Scores correlated – closer scores, more reliable PREM
• Patients must agree to taking part in ‘doing a PREM more than once a
year’ & have a patient ID number allocated in order to participate (in
order that their PREMs can be matched)

(3) Cognitive interviews
Do patients understand the questions in the way we expect them to?
>What does the patient think items are asking for?
>How do they retrieve the relevant information?
>What’s the decision making process involved in selecting an answer?
Interview 16 patients from 4 units in England – or more if there are any parts of
the PREM which haven’t been covered

Data Returns

Data Returns (Cont’d)

Feedback + next PREM!
• Overall report and 52 centre level reports available November 2016
• Validation completed February 2017
• 2nd
PREM Spring/Summer 2017

Richard Fluck
Clinical Co-Chair Internal Medicine
Programme of Care NHS England
Richard.fluck@nhs.net
Ron Cullen
Director
UK Renal Registry
Ron.cullen@renalregistry.nhs.uk
Paul Bristow
Director of marketing and
communications
British Kidney Patient Association
paul.bristow@britishkidney-pa.co.uk
How to find out more
Karen Thomas
Think Kidneys Programme Manager
UK Renal Registry
Karen.thomas@renalregistry.nhs.uk
Rachel Gair
Person Centred Care Facilitator
UK Renal Registry
Rachel.gair@renalregistry.nhs.uk
Sarah Evans
Think Kidneys Programme Coordinator
UK Renal Registry
Sarah.evans@renalregistry.nhs.uk
Contact Think Kidneys
www.linkedin.com/company/think-
kidneyswww.twitter.com/ThinkKidneys
www.facebook.com/thinkkidneyswww.
youtube.com/user/thinkkidneyswww.sli
deshare.net/ThinkKidneyswww.thinkkid
neys.nhs.uk
| 33Transforming Participation in Chronic Kidney Disease Rachel Gair

Developing and Implementing a Patient Reported Experience Measure

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to Developing and Implementing a Patient Reported Experience Measure (20)

More from Renal Association (20)

Recently uploaded (20)

Developing and Implementing a Patient Reported Experience Measure

Editor's Notes