How to Implement Quality in Health Care Organizations.
Download as PPTX, PDF6 likes1,505 views
The document discusses the challenges and methodologies of measuring and implementing continuous quality improvement (CQI) in healthcare settings, focusing on the need for proper structure and leadership. It outlines research questions and survey development aimed at assessing CQI implementation across various healthcare organizations, highlighting the results from a pilot study. The findings aim to provide hospitals with actionable insights and a roadmap for effectively improving healthcare quality outcomes.
How to Implement Quality in Health Care Organizations.
- 1. Measuring & Understanding Quality Improvement in Healthcare Mahboob ali khan, MHA / CPHQ, Consultant Quality - Operations Continental Hospitals Hyderabad, India
- 2. Continuing the Journey 1) Research Questions • Experience in Healthcare Operations 2) Theory / Models •Search and Study •Develop Questions and Hypotheses 3) Develop / Test •Develop 2 into something that can help answer 1 4) Examine Results • How does 3 answer 1 Focus Oral Exam Defense
- 3. Initial Research Questions Developed from 8 years of frustration in healthcare management knowing that I wasn’t equipped to provide appropriate support to clinicians What I Could Provide What I Needed to Provide Leadership Understanding of Healthcare Financial Direction Strategic Direction Management Capabilities A More Balanced Approach to Managing Assessment Data Management Study Design
- 4. Initial Research Questions How do I know something works? Continued requests for equipment, supplies and instruments Do patients get better? The End of Medicine Is there one way to do a procedure that is better than another? Significant variation in preference cards How do I measure quality?
- 5. Defining Quality IOM – The degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge. Donabedian - The systematic measurement and evaluation of the predetermined outcomes of a process, and the subsequent use of information to improve the process based on expectations of the customer.
- 6. Theories / Models – Oral Exam Ernest Amory Codman end results idea – 1920s Florence Nightingale (late 1800s) and Walter Shewhart (1920s) Statistical Process Control Edwards Deming, Joseph Juran and Kauru Ishikawa Continuous Quality Improvement (CQI) Avedis Donabedian structure – process – outcome Implicit vs. explicit criteria Everett Rogers Dissemination of Innovation Don Berwick, Paul Batalden, Brent James and Steve Shortell Recent literature
- 7. Structure – Process - Outcome Quality of healthcare can be assessed on the basis of structure, process (how care is delivered), and outcome (mortality, functional status, quality of life, and patient satisfaction) good measures of the first two are those that have a clear relationship to the third structure must proceed process which must proceed outcome Structure Definition Something arranged in a definite pattern of organization Organization of parts as dominated by the general character of the whole
- 8. Implementing CQI is Largely Structure Medical Scott CQI Outcomes Organization Division of labor HR / Training Specialty Mix Coordination Customer Focus Resources Training/Experience Planning Work Load Power Process Access Resources Innovation Buildings Supplier Partner Information Information Leadership Leadership Policies/Procedures Tasks
- 9. Structural Dimensions of CQI Implementation Strategic Cultural Technical Organiz. Result No Yes Yes Yes No significant results on anything important Yes No Yes Yes Small, temporary effects Yes Yes No Yes Frustration & False starts Yes Yes Yes No Inability to capture the learning & diffusion Yes Yes Yes Yes Lasting process change Adapted from Shortell et al. 1996
- 10. Translating Theory To Research – Putting Shape to my Frustrations 1. The healthcare system is broken The IOM reports 2. Appropriate structure – “the forgotten, but important component of the quality triad” – is wrongly assumed as present JCAHO 3. CQI, if implemented properly, can be the appropriate structure Managerial Philosophy 4. Healthcare providers are are finding it difficult to implement CQI Problem Solving Methodology
- 11. Research Question Why haven’t healthcare organizations been able to use CQI to differentiate themselves in terms of quality? Answer: Accountability & Assessment
- 12. Accountability Individual motivation has not been successful Midnight at the Waldorf-Astoria Rhetoric, not Reality (The Halothane Study) Large scale environmental change is needed Environment, Organization, Micro-System, Pt Current Motivators that may force change Patient Safety – Medical Errors “report cards” - PA, NY and CA Increase in Costs and Premiums Increase in the use of Alternative Medicine Variation in processes Increase in litigation
- 13. Assessment Issues & Research Questions 1. Low CQI knowledge level of senior leadership Do step by step instructions exist that assists researchers in studying CQI, and healthcare leaders in implementing CQI? 2. Implementing only a subset of the CQI domains Does a comprehensive survey and scale of CQI implementation exist? 3. Poor measurement strategies Are psychometrics examined appropriately in measuring CQI implementation? 4. Rhetoric does not equal the reality Does a measure exist that can be used to develop a quick snapshot of CQI implementation efforts in a hospital and is there a corresponding descriptive scale?
- 14. Step 1: Search the Literature Does a comprehensive, valid and easy to administer measurement tool exist that provides organization leaders with a descriptive scale and instructions for next steps? Methods - Used Ovid databases with a focus on business, psychology, sociology and healthcare Keywords: measurement quality, continuous quality improvement, total quality management, implementation Scrolled through quickly at least 10,000 references most focused on implementing only a few domains of CQI Articles not deleted included: Measuring Quality; Domains and Implementation of Quality
- 15. Quality Overload Using Medline through PubMed from 1995 to the present / English only ‘quality improvement’ = 8,848 ‘continuous quality improvement’ = 1,100 ‘quality’ in title = 17,466 ‘quality improvement measure’ in title = 3 from 2000 to present / English only ‘quality improvement’ in title = 350 50 usable, 30 ‘easily findable’, 15 good, 1 measurement
- 16. Worldwide Measurement for QA/QI Structure AWARDS - too time intensive, gold standard domains The Malcolm Baldrige Award Leadership, HR/Training, Process, Business Results, Customer Focus, Information Systems, Planning, Partnership EFQM US State-Based Awards ACCREDITATION & CERTIFICATION - questions on validity and too time intensive JCAHO IS0 9002 - 2000 Inspection, Contract, Public Responsibility, Innovation, Product Control, Servicing SURVEY INSTRUMENTS
- 17. Survey Instruments Eleven worldwide surveys examining CQI as a managerial philosophy were analyzed (Tables - pges 1 & 2) 8 from the US, 1 from Canada, the Netherlands, and Australia 5 were specific to healthcare, including the ‘gold standard’ from Shortell et al. Most examined psychometrics while very few provided a scale The shortest (22 questions) was also the least comprehensive The Baldrige Domains dominated
- 18. Survey Instruments Weaknesses of the current surveys included: Relatively few domains other than the Baldrige were even mentioned The most comprehensive surveys are much too long (depth vs. breadth) Strong potential for respondent bias if survey is only given to one level employee in an organization Very few surveys provided a scale Published in the International Journal for Quality in Health Care 2001: Volume 13, Number 3: pp 197-207
- 19. Step 2: Develop and test a comprehensive and concise measure of CQI implementation Initial Survey 14 domains (Baldrige, EFQM, ISO) and 70 items, with each domain containing at least 4 items All but a very few items were from the 11 surveys analyzed in Step 1 Items were chosen by the researchers using a subjective analysis and whether or not the question could be answered using a 5 point Likert scale
- 20. Content Validity The benefits of a content validity study for this study True experts in the field of CQI Past measures have gone through psychometric testing Excellent method of data reduction Methods Statistical method described in Grant & Davis (1997) and Lynn (1986) 1. Panel of Experts All either attend an invite only CQI symposium sponsored by Dartmouth, have recently taught CQI at a Masters level or are positional leaders of QI efforts in a healthcare organization Best to have between 7 and 10 - this study had 7 from the US and 1 from England
- 21. Content Validity Methods (cont.) 2. Scoring Grid (See Sample Grid - pge 3) Each expert was emailed the scoring grid with definitions and instructions. Is the item clear and understandable? 4 point scale Does the item represent CQI? 4 point scale Match the item with a domain. 1 through 14 representing each domain & 15 representing unable to classify
- 22. Content Validity Methods (cont.) 3. Indices Inter-rater agreement (IR) = # of raters who scored an item as high / total # of raters high defined as a 1 or 2 on both 4 point scales acceptable IR > / = .70 Content Validity Index (CVI) = # of items where all experts rated high / # of items acceptable CVI > / = .80 Domain congruence = % of time where experts chose the same domain as the investigators
- 23. Content Validity Results After 4 analysis iterations where poorly rated items were deleted, the questionnaire included: 22 items 8 domains Clarity IR of .91 (range of .85 - 1) Representativeness IR of .93 (range of .87 - 1) Clarity CVI of .73 using Lynn’s (1986) method Representativeness CVI of .91 using Lynn’s (1986) method Overall, the experts chose the same domain as the investigators in the original measure 76% of the time
- 24. Content Validity Results Investigators added 6 items to ensure that every domain except for Supplier Partnership contained 3 items Investigators changed some wording to increase clarity Final survey was sent back to experts for comments Other Results The scoring grid took a lot longer to complete than originally thought The leadership domain had the highest rate of agreement QI must be differentiated from QA Baldrige criteria dominate
- 25. Step 3: Develop a corresponding scale of CQI implementation A 5 level corresponding scale (pge 4) was developed by the investigators from: Roger’s Diffusion of Innovations agenda setting, matching, redefining / restructuring, confirmation, clarifying, routinizing Deming’s Continuous Quality Improvement match domains with appropriate level Samsa & Matchar CQI as a problem solving methodology vs. a managerial philosophy Characteristics: CQI is a developmental process Time is important Scale provides focus for future quality initiatives
- 26. Steps 2 & 3 - Content Validity and Scale Published and Voted Best Student-Led Paper in the 2002 Business and Health Administration Proceedings,pges 198-204 Will be further published in Either Quality in Health Care or Hospital Topics
- 27. Step 4: Is the survey and scale easy to use and are the results practical? Pilot Study Worked with the Missouri Hospital Association for contacts. 83 Missouri hospitals eligible (above 40 beds), 40 participated 5 responses from each hospital: CEO/COO, Director of Quality, a non-salaried MD, and 2 managers Survey and results disseminated via email Hypotheses based on Paper 1 Weaknesses 1. There will be measurable differences between and within hospitals. 2. The survey will have high known-groups validity. 3. The items and domains will differentiate between levels as hypothesized by the conceptual scale.
- 28. Pilot Study Methods Known Groups Validity Examined the relationship between the survey and: state quality team winners >/= Level 2 state quality organization winners >/= Level 3 national quality award finalists >/= Level 3 subjective quality assessment at 10 of the 40 hospitals question 1 (pge 5) asking the participants to categorize their quality structure Reliability Cronbach’s Alpha for each domain, each title, and the overall measure
- 29. Pilot Study Methods Between Hospital Variation One-way ANOVA & Bonferroni by size, region and ownership model Within Hospital Variation Repeated Measures ANOVA & Bonferroni by title Item and Domain Analysis ANOVA & Bonferroni determine which items and which domains discriminate well between different levels of the scale
- 30. Pilot Study Hospital Total Level N = 40 (min of 90, max of 130) 16 14 12 10 8 6 4 2 0 Level 1 (</= 100) Level 2 (101-110) Level 3 (111-120) Level 4 (121-130) Level 5 (131-140) # of hospitals
- 31. Pilot Study Results Sample Characteristics (pge 6) 40 hospitals are significantly larger and more likely to be for profit and part of a system Of the 200 returned surveys, there was less than 5% missing values and ‘I don’t know / NA’ Known Groups Validity 2 of the 12 did not score as hypothesized hospitals were not significantly different than others 4 of 9 (44%) similar for the subjective assessment 40% agreement for question 1 assessment
- 32. Pilot Study Results Reliability Cronbach’s Alpha ranged from .54 (HR/Training) to .84 (Innovation) for the domains Information .69; Process, Planning .77; Customer .78; Leadership .83 Cronbach’s Alpha ranged from .88 (Director /Manager and QI Director) to .92 (MDs) for different groups Cronbach’s Alpha was .94 for the overall measure
- 33. Pilot Study Results Between Hospital Variation Region was only attribute that was significant Within Hospital Variation Senior Executives significantly lower than QI Directors QI Directors significantly higher than Managers / Directors MDs significantly higher than Managers/ Directors supports surveying more than one level employee
- 34. Pilot Study Results Item and Domain Analysis (pge 7) Of the 28 items, 4 did not show good differentiation between any of the levels these should be either reworded or changed Leadership showed significant differentiation between all levels studied Planning showed significant differentiation between 2 of the 5 levels
- 35. Pilot Study Results Quality Improvement Scale Level Components after Domain Analysis 1. Quality Assurance 2. QI Low High Focus – Leadership ( visibility) ; Customer Focus M edium Focus – I nnovation 3. QI Medium High Focus – Process; HR / Training; Planning M edium Focus – Leadership ( support) 4. QI High M edium Focus – I nformation; Supplier Partnership Low Focus – Leadership ( consistency) ; Planning 5.QI - Absorbed
- 36. Findings The survey is easy to administer The survey provides a reliable and valid snapshot of CQI implementation in a healthcare organization No known group exists The scale is a practical method of providing hospital leaders with a roadmap for CQI implementation Leadership is the most important component of implementing CQI Submitted to Health Services Research
- 37. A Likely Future Scenario 1. Patient Safety provides accountability to analyze quality and outcomes 2. Healthcare leaders see CQI as a methodology to improve patient outcomes 3. Hospitals use the survey and scale to help assess & implement CQI appropriately, which in turn eliminates structure issues discussed 4. Because of this, hospitals can effectively assess their processes and improve their outcomes
- 38. Future Research Questions Can a clearer snapshot of CQI implementation emerge using line worker responses, and senior leadership interviews? Administered the survey to a 40 random line workers at 10 hospitals Initial results include: 50% return rate high number of ‘I don’t know / NA’ responses all hospitals overall employee score < 100 (Quality Assurance) Senior hospital leadership meetings to discuss quality structure Initial results include: low knowledge of ‘quality’ among the senior leaders structures developed with little statistical or facilitation resources
- 39. Future Questions Does a higher level of CQI implementation lead to better financial, quality and safety outcomes? If not, why? Develop and Find Financial, Operational and HR Effectiveness and Efficiency Measures Counte & Glandon, 1995 Build one, clean database with CQI implementation scores and measures Analyze to assess relationships
- 40. Future Questions Is CQI, as its described in the literature, an effective method for improving quality outcomes? Change the four items that did not differentiate well 1. How many multi-disciplinary teams currently work to improve the processes of care in your organization? (Process) more statistics, less teams • Human Factors Research • Toyota • Six Sigma 2. Do people in your organization know who their customers are? (Customer Focus) not clear 3. Are employees in this organization encouraged to try new and better ways of doing things? (Innovation) 4. Is creativity actively encouraged in this organization? (Innovation) healthcare has typically not been innovative and so these questions may need to be more specific
- 41. THANK YOU