Artificial Intelligence in Healthcare: A Change Management Problem
Download as PPTX, PDF1 like2,984 views
The document discusses the role of artificial intelligence (AI) in healthcare, emphasizing that successful implementation relies on effective change management rather than just technical accuracy of predictive and prescriptive models. It outlines a framework consisting of three levels of understanding—functional, contextual, and operational—to facilitate the adoption of AI technologies and improve healthcare outcomes. The importance of collaboration among data scientists and clinicians is highlighted to ensure the models are trustworthy and usable.
Artificial Intelligence in Healthcare: A Change Management Problem
- 1. Artificial Intelligence in Healthcare: A Change Management Problem Health Catalyst Editors
- 2. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Jason Jones, PhD Chief Data Scientist Health Catalyst This report is based on a webinar presented by Jason Jones, PhD, Chief Data Scientist at Health Catalyst February 27, 2019, titled, “3 Perspectives to Better Apply Predictive & Prescriptive Models in Healthcare.” Artificial Intelligence in Healthcare
- 3. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Artificial Intelligence in Healthcare Artificial intelligence (AI) is a hot topic in health and healthcare today. Most of the focus has been on the technical challenges of building and deploying predictive and prescriptive models. While there is always room for improve- ment, algorithms have advanced to a point that allows us to focus on the bigger challenge: change management. By acknowledging that change is hard, “tooling up,” and maintaining a human- centered perspective, we can achieve better adoption and results with AI.
- 4. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Artificial Intelligence in Healthcare Predictive and prescriptive models can enhance health and healthcare, which will ultimately help improve Quadruple Aim outcomes: population health, patient experience, reduced cost, and positive provider work life. However, successful adoption of predictive and prescriptive models heavily depends upon behavior change, which requires more than technical accuracy.
- 5. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Artificial Intelligence in Healthcare While predictive algorithms abound, tools to facilitate change management remain scarce. This article will help clinical and operational leaders obtain value from predictive and prescriptive models using three perspectives: functional, contextual, and operational. Investing time and effort to ensure these three levels of model understanding is necessary for broad-scale AI adoption.
- 6. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Predictive Versus Prescriptive Models Healthcare leaders currently delving into AI must understand what predictive and prescriptive models are and how they differ: Data scientists try to anticipate a person’s behavior or a circumstance before it happens—ideally, with enough time to act—but it does not necessarily tell data scientists what action to take. For instance, a predictive model might reveal a patient’s prognosis and a 30 percent chance of readmission within 30 days (but not how to manage the patient’s risk). Predictive Model Not only does the model tell data scientists what is going to happen, it advises the specific course of action that should be take in response. In the readmission example, a prescriptive model would reveal the prognosis and readmittance odds but would advise a phone call to the patient one day after discharge and a multidisciplinary visit three days after discharge. Prescriptive Model
- 7. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Artificial Intelligence in Healthcare Healthcare leaders and data scientists have many predictive and prescriptive modeling resources available, including some free clinical prediction models. In fact, these resources teach healthcare professionals that developing predictive models is no longer a question of technical aptitude but whether a health system is prepared to deploy them.
- 8. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. A Framework for AI Change Management The first question clinical and operational leaders need to answer is, “What are we trying to achieve?” Algorithms and predictive models are commodities employed to achieve specific outcomes. It’s important to know the desired outcome before setting out to build a predictive model. If you don’t know where you’re going, any road will take you there.” - George Harrison
- 9. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. A Framework for AI Change Management Healthcare leaders often disagree over the best course of action to affect change within an organization. To successfully adopt AI technologies, frontline leaders can use a framework for change management that sets expectations and facilitates productive dialogues to encourage consensus and buy-in. The framework includes three levels of understanding: 1. Functional 2. Contextual 3. Operational
- 10. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. A Framework for AI Change Management 1. Functional: Does the model make sense? If a patient has multiple comorbid conditions, does it make more sense that he’s going to be readmitted, or that he’s taken care of because he’s receiving care from a group of specialists? The functional level of understanding ensures data scientists and healthcare leaders recognize the relationship between the input feature or parameter and what they’re trying to predict or suggesting as an action item.
- 11. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. A Framework for AI Change Management 2. Contextual: How does it fit into the workflow? A healthcare analytics team has built the world’s most accurate predictive model, but the team doesn’t know what to do with the results. Analytics teams can’t lose sight of the fact that the results of this prediction will drive patient action. It is imperative that the team understands who the patient is, what she’s going to do with the prediction, and when it would make sense for her to do it.
- 12. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. A Framework for AI Change Management 3. Operational: What benefits and risks are traded? Healthcare professionals and clinicians make very few choices that are risk-free and don’t require some level of trade-off. How can analytics teams understand those trade-offs and make the best decisions for the organization? And how can the organization make those trade-offs with eyes wide open and accurately measure if they’re achieving success?
- 13. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Levels of Predictive Model Understanding Once healthcare leaders have the three- level framework for change management, they can begin to deploy predictive and prescriptive models. These three levels of understanding create the framework for change management that enables successful deployment and adoption of predictive and prescriptive models.
- 14. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Levels of Predictive Model Understanding Attaining functional, contextual, and operational understanding helps ensure that teams don’t lose track of the problem they’re trying to solve and bolsters their confidence within the organization to move decision makers toward agreement.
- 15. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding To demonstrate, this article will examine three examples of how teams can move through the three levels of understanding. Functional Model Understanding Contextual Model Understanding Operational Model Understanding
- 16. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Functional Model Understanding In the first example, an analytics teams wants to build a predictive model that determines which patients in the emergency department (ED) are at greatest risk for progressing to severe sepsis or septic shock within 24 hours.
- 17. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Functional Model Understanding In functional model understanding, the team might begin by asking questions to achieve a basic understanding of how the model should work: Does a patient’s risk for severe sepsis or septic shock increase when his temperature is low or high? If the patient’s temperature is low, should his risk go up or down? If the patient’s temperature is high, should his risk up or down? If so, by how much? > > > >
- 18. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Functional Model Understanding The analytics team needs help answering these questions; they need input from clinicians to build a model that makes sense. Figure 1 illustrates a sample model for predicting whether a patient will progress to severe sepsis or septic shock within a 24- hour period, using six key parameters. In this example model, the patient’s risk of developing sever sepsis or septic shock moves up or down based on how the parameters change.
- 19. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Functional Model Understanding Figure 1: Functional model understanding—sample model for predicting whether a patient will progress to severe sepsis or septic shock within a 24-hour period.
- 20. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Functional Model Understanding Clinician input and understanding are key to getting the healthcare organization to embrace and adopt this model down the road. Data analysts and data scientists need to make the predictive model transparent, so clinicians understand how it works, and, in turn, support the model—a critical step for adoption.
- 21. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Functional Model Understanding George Box, a British statistician, famously said, “All models are wrong. Some models are useful.” The team is trying to build a useful model, and the utility will not just be how well it predicts the event, but what somebody does with the newly discovered information. Obtaining functional model understanding is the first step.
- 22. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Contextual Model Understanding Taking the Sepsis example above, obtaining contextual model understanding requires the team to ask a different set of questions: Are we finding the right patients at the right time? Who should take action? > >
- 23. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Contextual Model Understanding The next question the team might ask for contextual understanding is, “Who’s going to take action?” Should a phlebotomist draw a lab with an automated order? Should the patient receive fluids earlier? Figure 2 depicts an example I-Chart that provides details leading up to an incident that can be used for contextual model understanding.
- 24. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Contextual Model Understanding Figure 2: Sample I-Chart that provides contextual model understanding.
- 25. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Contextual Model Understanding To obtain contextual model understanding, the team would look at patients who may be at risk for severe sepsis or septic shock. The I-Chart provides additional information about diagnoses, comorbidity, and recent clinical encounters (e.g., whether the patient had recently been in an ambulatory setting). The chart shows the first 24 hours, from the time the patient checks into the ED, and the six parameters identified above by the analytics team in functional model understanding.
- 26. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Contextual Model Understanding This allows the team to understand the complete context for this patient in relationship to the predictive model. With this information, and the questions they ask in the proper context, the team can make adjustments to improve the model’s utility and further gain clinician buy-in.
- 27. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Operational Model Understanding Using a different example to demonstrate operational model understanding, the team wants to identify patients who will develop congenital heart disease (CHD). Unlike the sepsis example, the team is not limited to events in a specific set of hours or days but decades worth of information.
- 28. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Operational Model Understanding In building this model, the team will want to consider the following questions: What percentage of patients will the model capture? If the patient is prescribed a statin, what is the chance she will develop CHD? > >
- 29. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Operational Model Understanding To build a useful model, the team will not only need to think about which patients might develop heart disease over the next decade but also what to do with that information. One possible action is to prescribe a statin for the patient. Prescribing a statin is a relatively low-intensity action but also comes with possible side effects and associated costs. Operational model understanding compels teams to examine the cause and effects and weigh the tradeoffs that often come with medical decision making.
- 30. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Operational Model Understanding If the team’s goals for this model are to capture 50 percent of patients who develop CHD in the next 10 years and, for those patients identified, have 50 percent of statin prescriptions associated with CHD, how can they set expectations appropriately for the tradeoffs they’re making?
- 31. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Three Example Predictive Models to Demonstrate Understanding Operational Model Understanding Can they gain clinician and leadership buy-in for a model that would benefit 50 percent of identified patients? If not, how can they modify the model to get the buy-in they need for adoption? This type of organizational dialogue and value balancing facilitates operational model understanding.
- 32. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Supporting Leadership Decision Making To successfully adopt predictive and prescriptive models, most healthcare organizations will do the heavy lifting in change management and efforts to overcome organizational inertia. The analytics teams can leverage three specific tool sets to facilitate the change management process: functional understanding, contextual understanding, and operational understanding. With these three tools, teams are more likely to successfully deploy predictive and prescriptive models.
- 33. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Supporting Leadership Decision Making An important part of the success of this model is ongoing collaboration, along with the goal of creating something that everyone (from data scientists to clinicians) can understand, trust, and utilize. Successful deployment ultimately depends on the usefulness and level of adoption of the model.
- 34. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. For more information: “This book is a fantastic piece of work” – Robert Lindeman MD, FAAP, Chief Physician Quality Officer
- 35. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. More about this topic Link to original article for a more in-depth discussion. Artificial Intelligence in Healthcare: A Change Management Problem Meaningful Machine Learning Visualizations for Clinical Users: A Framework Valere Lemon, MBA, RN, Senior Subject Matter Expert; Alejo Jumat, User Experience Designer, Sr. Healthcare Data Management: Three Principles of Using Data to Its Full Potential Sean Whitaker Machine Learning in Healthcare: What C-Suite Executives Must Know to Use it Effectively in Their Organizations Eric Just, SVP and General Manager, Product Development; Levi Thatcher, VP, Data Science Tom Lawry, Director, Worldwide Health, Microsoft A New Era of Personalized Medicine: The Power of Analytics and AI Health Catalyst Editors Machine Learning, Predictive Analytics, and Process Redesign Reduces Readmission Rates by 50 Percent Health Catalyst Success Stories
- 36. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Other Clinical Quality Improvement Resources Click to read additional information at www.healthcatalyst.com Jason Jones currently serves as Chief Data Scientist at Health Catalyst. Previously, he served at Kaiser Permanente (KP) in various roles including Research Scientist and VP, Information Support for Care Transformation. Prior to KP, Jones was a Medical Informaticist for Intermountain Healthcare. Other roles have included analytic and marketing leadership positions at Bayer HealthCare, data and information product development at UnitedHealth Group, and various academic adjunct faculty positions. Jones received his PhD in Biostatistics from the University of Southern California in 2001. His mission is to leverage data to achieve the Quadruple Aim. Jason Jones, PhD
- 37. © 2020 Health Catalyst Proprietary. Feel free to share but we would appreciate a Health Catalyst citation. Other Clinical Quality Improvement Resources Click to read additional information at www.healthcatalyst.com Health Catalyst is a mission-driven data warehousing, analytics and outcomes-improvement company that helps healthcare organizations of all sizes improve clinical, financial, and operational outcomes needed to improve population health and accountable care. Our proven enterprise data warehouse (EDW) and analytics platform helps improve quality, add efficiency and lower costs in support of more than 65 million patients for organizations ranging from the largest US health system to forward-thinking physician practices. Health Catalyst was recently named as the leader in the enterprise healthcare BI market in improvement by KLAS, and has received numerous best-place-to work awards including Modern Healthcare in 2013, 2014, and 2015, as well as other recognitions such as “Best Place to work for Millenials, and a “Best Perks for Women.”