Measurement for Improvement
Download as PPT, PDF0 likes845 views
The document discusses the importance of measuring performance to drive improvement. It outlines a 7-step process for establishing effective measurement: 1) decide the aim, 2) choose measures, 3) define measures, 4) collect data, 5) analyze and present data, 6) review measures, and 7) take action on results. Key points include measuring outcomes and processes, collecting consistent data, analyzing variation over time through run charts, and using measurement to identify improvement opportunities.
Measurement for Improvement
- 1. Measuring for Improvement NECLEC HIEC 22 nd June 2011
- 2. Why measure? Deciding what and how to measure
- 3. What is an innovation? What is an improvement?
- 4. Why should we measure? “ All improvements involve a change, but not all changes are improvements” Goldratt
- 5. “ You can’t fatten a cow by weighing it” (Palestinian proverb) but…….. How do we know if a change is an improvement? “If you can’t measure it, you can’t prove it” Measurement for improvement
- 6. Why is information not used?
- 7. 7 Steps to measurement
- 8. Step 1 – Decide Aim
- 9. Aims exercise If you were in a lift with the rest of your table group could you clearly and briefly describe your aim in a sentence – i.e. the time it takes to travel from one floor to the next? What’s the aim of the project? Try it! Are your colleagues aims clear and understandable to you?
- 10. Step 2 – Choose Measures
- 11. Choosing measures – weight loss example Aim: 2 stones lighter! Energy Out Energy In Walk daily commute Stairs not lift Exercise Reduce alcohol intake Eat Less Pedometer Gym work out 3 days Squash weekends No pub weekdays Take packed lunch Low fat meals
- 12. What data do you collect routinely to understand your elective caesarean section pathway? What measures do you use?
- 13. Step 2 – Where to measure? Start ? Decision Point ? Handover ? End ?
- 14. Step 2 – What to measure? Patient experience and outcomes Audit of patient experience of the labour ward environment % of women who contribute to their birth plan Safety and reliability Audit of post-operative infection rates Audit of practice against VBAC guidelines Efficiency and value Percentage of women who are discharged on the planned date Audit of delays against discharge plan Leadership and high performing teams Recruitment, retention and sickness absence rates % of clinical staff aware of monthly CS rates and trends
- 15. Types of measures Measuring the effectiveness of your pathway Outcome measure – what is the result? Measuring the efficiency of your pathway Process measure - Are the parts in the process performing as planned “ Every system is designed to get the results it gets”
- 16. Step 3 – Define Measures An operational definition is a description, in quantifiable terms, of what to measure and the steps to follow to measure it consistently Are we measuring the same thing?
- 18. Step 4 – Collect Data Practical considerations: 5 W’s and 1H
- 19. Step 4 – Collect Data Decisions, decisions… What - All patients or a sample? Who – took the data? (what role?) When – When was the data taken - real time or retrospective? Where is the data from? How – was the data taken - hospital system or audit? Turn the data into a different unit (hours into days)
- 20. Step 5 – Analyse & Present The type of presentation you use has a crucial effect on how you react to data and how you can identify variation “ Data should always be presented in such a way that preserves the evidence in the data…” Walter Shewhart
- 21. What does this data tell us? Patients seen in April 2008 2009 600 550 610 540 560 570 580 590
- 22. Are we assuming something like this? A linear trend
- 23. Or this? NO trend
- 24. But it could be something like this? Seasonal dip
- 25. Or this! One month blip
- 26. What does this data tell us? This Month Last Month Given two different numbers, one will always be bigger than the other! Something Important What action is appropriate? “ When you have two data points, it is very likely that one will be different from the other” W. Edwards Deming
- 27. Change implemented in June Is this familiar? Are you familiar with looking at data like this? There is a tendency to develop guesswork on our part on what the data is showing us. We tend to make decisions based on single figures. But how effective is this? What can we gather from this table? How useful is the data represented like this? Remember that a picture is worth a thousand words and much easier to read than tables. Date Average Length of Stay (Hours) Jan 70 Feb 68 Mar 67 Apr 68 May 69 Jun 58 Jul 56 Aug 55 Sep 57 Oct 54 Nov 56 Dec 55
- 28. Same data presented as a run chart Target – 56 hours Change implemented in June The same data presented as a line graph or run chart shows the impact of change and allows you to see variation over time
- 29. Step 5 – Analyse & Present SPC Run Chart
- 30. Plotting the dots - example Run Chart Number of Emergency admissions for COPD (weekly) November 2009 to June 2010 0 No of Calls 180 160 140 120 100 80 60 40 20 1 st Nov 15 th Nov 29 th Nov 13 th Dec 27 th Dec 10 th Jan 24 th Jan 7 th Feb 21 st Feb 6 th Mar 20 th Mar 3 rd Apr 17 th Apr 1 st May Week Calls per week Median
- 31. The Myth of Trends Upward trend ? Downward trend ? Downturn ? Setback ? Turnaround ? Rebound? Static ? Flatline ?
- 32. Looking for a trend 7 points all in upward direction 7 points all in downward direction Time Downward trend Time Upward trend
- 33. Looking for a trend 7 points above centre line 7 points below centre line Time Below centre Time Above centre
- 34. Step 6 – Review Measures “ It is a waste of time collecting and analysing your data if you don't take action on the results “
- 35. 7 Steps to Measurement You may not get it right first time! You may need several iterative attempts through steps – 4 - 6
- 36. Telephone triage
Editor's Notes
- #4: So why should we measure? It is important to understand where you are now, where you want to get to and to make informed decisions about how you are going to get there. These quotes relate to two important aspects of measurement. Firstly, for any improvement work you undertake you need to know that change have made a real difference. As the first quote shows all improvements involve a change, but not all changes are improvements. The only way we will know if a change has led to improvements is to measure. Secondly, we have a tendancy to compare month against month figures and if next month you have gone up you have made an improvement or vice versa. As the quote states, when you have two data points, it is very likely that one will be different from the other. So we need a more sophisticated (but not more complex!), meaningful way to see how we are performing which this presentation will explore.
- #5: So why should we measure? It is important to understand where you are now, where you want to get to and to make informed decisions about how you are going to get there. These quotes relate to two important aspects of measurement. Firstly, for any improvement work you undertake you need to know that change have made a real difference. As the first quote shows all improvements involve a change, but not all changes are improvements. The only way we will know if a change has led to improvements is to measure. Secondly, we have a tendancy to compare month against month figures and if next month you have gone up you have made an improvement or vice versa. As the quote states, when you have two data points, it is very likely that one will be different from the other. So we need a more sophisticated (but not more complex!), meaningful way to see how we are performing which this presentation will explore.
- #6: Measurement is important, but its not the whole project
- #8: 7 steps to measurement Say that measurement doesn’t just happen. It requires a well defined process. The diagram shows the 7 steps necessary to get measurement to work for you. Move to next slide For reference Step 1 – Decide your aim Step 2 – Choose your measures Step 3 – Confirm how to collect and display your data Step 4 – Collect your baseline data Step 5 – Analyse and present your data Step 6 – Meet to decide what it is telling you Step 7 – Repeat steps 4 to 6 each month or more frequently
- #13: Group discussion / flip charts
- #16: So what types of measures should we be collecting? The obvious measures which people tend to routinely collect are outcome measures, which measure how effective your pathway is. For example, CS rates, and patient satisfaction experience. It is vital that you also collect data on how your pathway/process is performing that has resulted in those outcomes for the patient. So you know from mapping your c-section pathway this morning what the steps are in your process. How do you know how those steps are performing and what input they have to contributing to the end outcomes. There is a phrase “every system is designed to get the results it gets” This means purely that your current process map is desigend to get the outcomes that you have whether it be your CS rate, length of stay for a woman after a CS. Without redesign, the same system and processes that have created the current reality will work together to repeat it, even if a single participant is removed. If we want different performance or outcomes we must change the system/
- #18: Its not just about collecting data, but more importantly what you do with the data to turn it into meaningful information which will help support improvements to your service. In the toolkit, there is a section specific to Measures and some examples of what you can collect for each of the pathways. This is not a definitive list of measures, these are just examples/suggestions. For this project,,,,
- #28: To get a clearer understanding of how you are doing, you need to collect data over a series of time. The table shows the average length of stay for an elective CS over a period of 12 months. From this you can already start to see a difference in LOS in June where a change was implemented. How can we transfer this data into a more meaningful way?
- #29: A run chart is a simple line graph which shows your performance over time, when you want to look for trends and patterns over time, or if an implemented change is making things better, worse, or having no effect. To ensure that run charts are interpreted correctly, keep a record of external factors and events that may influence the outcomes such as when a clinician is absent because of illness.
- #31: !! ANIMATED SLIDE !! Slide click to add axis, axis labels, chart title, chart key and then median line Draw attention to these…
- #32: Given any three points we could make the following 6 types of analysis, none are valid, they are not statistically significant.
- #33: A run of 7 points is. All up, down or all above or below a centre line.
- #34: A run of 7 points is. All up, down or all above or below a centre line.
- #35: Group discussion Report & review (10 minutes for slides 18-19) Explain they are now going to decide where the review meeting(s) will happen As a whole group, use a flip chart (the facilitator may want to scribe at this point) and discuss how they will use the measures information: i) how it will be reviewed and communicated – e.g. in a theatre user group, newsletter, theatre display board ii) when will you review these – for each of the methods identified in (i) , how frequent will you do this Show an example of a review meeting as set out in slide 19.