Data Science Salon: nterpretable Predictive Models in the Healthcare Domain
1 like634 views
The document discusses the development of interpretable predictive models in the healthcare domain, specifically for behavioral health, utilizing extensive data extraction and feature engineering. Various modeling techniques were applied to create a behavioral health severity score combined with a medical score, enhancing the ability to direct individuals to appropriate care. The presentation also highlights the importance of interpretable models for fostering trust, detecting bias, and driving follow-up actions in healthcare settings.
Data Science Salon: nterpretable Predictive Models in the Healthcare Domain
- 1. Interpretable Predictive Models in the Healthcare Domain Sridharan Kamalakannan 2/22/2019
- 2. Pipeline DATA • Demographic • Medical & Claims • Pharmacy Claims • Lab & Test Results • Welcome Calls • Health Programs • Consumer Data PREDICTOR CATEGORIES Demographics Clinical Behavioral Medication E.g., Age, sex disability E.g., chronic conditions, BH, hospitalization, screenings Lifestyle, health programs E.g., asthma, diabetes, heart failure, BH MODELING CHOICES BH Severity Predictive Model Data Extraction Feature Engineering and Transformation Over 4,000 potential features were used for development. Model Development • Selection of the most important predictors • Different modeling techniques and combinations explored 1 2 3 • Branching • Linear Regression • Decision Trees • Neural Networks • Least Angle Regression • Ensemble
- 3. Application The BH severity score can be paired with a medical score (Charlson Comorbidity Index) to create an analytic framework called the Behavioral Health Quadrants, allowing: • Better understanding of the relative severity of medical & behavioral health in an individual • Greater ability to direct individuals to the right level of care 28% of Medicare Advantage participants had sufficient BH utilization to be assigned to a BH quadrant. Less Medical Health Risk/Complexity More Severe Med + BH At Risk Severe BH + Severe Med Severe BH + Mild-Moderate Med BH At Risk + Mild-Moderate Med LessBehavioralHealth Risk/ComplexityMore *Adapted from Four Quadrant Clinical Integration Model (National Council for Behavioral Health). Values as of Jan 2016. Based on membership as of Dec 2015 and claims Jan 2015 – Dec 2015. Medicare Advantage BH Quadrants*
- 4. Agenda 01 | An Example 02 | Why are interpretable models necessary? 03 | Taxonomy of interpretable models 04 | LIME Intuition 05 | LIME Recipe 06 | Pros and Cons 07 | Humana Use Cases 4
- 5. An Example: Husky vs Wolf Classifier Husky • References to the above picture are at the end of the presentation • https://arxiv.org/pdf/1602.04938.pdf Wolf ?
- 6. Click to edit Master title style Human Curiosity Satisfy the natural human curiosity to find explanations for unexpected events Trust Build enough trust in the model so that it is accepted and used in the real-world Detecting Bias Unethical discrimination of models against certain factors Identify leaks High accuracy can sometimes be due to leakage from the target Prescriptive Analytics Model explanations drive/prescribe follow up actions items Audit & Improve Explanations serve as a debugging tool to continuously audit and improve the model Why are interpretable models necessary? https://christophm.github.io/interpretable-ml-book/lime.html https://arxiv.org/pdf/1602.04938v1.pdf
- 7. Taxonomy of interpretable models Intrinsic vs Post-hoc Model-Specific vs Model Agnostic Local vs Global https://christophm.github.io/interpretable-ml-book/lime.html https://arxiv.org/pdf/1602.04938v1.pdf
- 8. LIME : Intuition The key intuition behind LIME approach is that it is much easier to approximate a black-box model by a simple model locally (in the neighborhood of the prediction we want to explain), as opposed to trying to approximate a model globally. https://christophm.github.io/interpretable-ml-book/lime.html https://arxiv.org/pdf/1602.04938v1.pdf
- 9. LIME Recipe 1 • Select your instance of interest for which you want to have an explanation of its black box prediction 2 • Perturb your dataset 3 • Get the black box predictions for these new points 4 • Weight the new samples according to their proximity to the instance of interest 5 • Train a weighted, interpretable model on the dataset with the variations 6 • Explain the prediction by interpreting the local model M X Y Simple Linear Model Decision Tree TOP DRIVERS W https://christophm.github.io/interpretable-ml-book/lime.html https://arxiv.org/pdf/1602.04938v1.pdf
- 10. Pros and Cons Pros Model-Agnostic Feature-Agnostic Widely employed by ML solutions Cons Computationally complex Sensitive to neighborhood
- 11. Smoothen Outreach Humana Clinicians are equipped with talking points that are top drivers of our members’ health – smoothing an initial outreach Produce the 5 most clinically-relevant health drivers per member and display for the clinician at time of referral Identify Top Barriers Interpretable models help identify the top barriers that impede a person’s Disease Specific Best Practices Focused Interventions Identifying the barriers helps clinicians focus on addressing important and relevant health needs Humana Use Cases
- 12. Questions?
- 13. References • https://www.google.com/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjDh4ve- N7fAhWOT98KHcbvCb8QjRx6BAgBEAU&url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FSiberian_Husky&psig=AOvVaw16MpnkTNQEnLZG8vvACIHo&ust=1547062800332581 • https://www.google.com/imgres?imgurl=https%3A%2F%2Fwww.pointvicentevet.com%2Fsites%2Fdefault%2Ffiles%2Fstyles%2Flarge%2Fadaptive-image%2Fpublic%2Fsiberian-husky-dog-breed- info.jpg%3Fitok%3DynYUSa8o&imgrefurl=https%3A%2F%2Fwww.pointvicentevet.com%2Fservices%2Fdogs%2Fbreeds%2Fsiberian-husky&docid=PnpomRMFvij6zM&tbnid=J34VKj- dw7vPcM%3A&vet=10ahUKEwjejrGW-N7fAhXoRd8KHTwBAiwQMwhtKAQwBA..i&w=480&h=480&bih=955&biw=1920&q=husky&ved=0ahUKEwjejrGW- N7fAhXoRd8KHTwBAiwQMwhtKAQwBA&iact=mrc&uact=8 • https://www.google.com/imgres?imgurl=https%3A%2F%2Fi.redd.it%2Fpncd4uz5q4q01.jpg&imgrefurl=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhusky%2Fcomments%2F8a1ybo%2Fso_sweeet_ 3%2F&docid=I3qu1cH71l6bCM&tbnid=zjKrKoKEnCd6uM%3A&vet=10ahUKEwjejrGW- N7fAhXoRd8KHTwBAiwQMwi7AShFMEU..i&w=220&h=261&bih=955&biw=1920&q=husky&ved=0ahUKEwjejrGW-N7fAhXoRd8KHTwBAiwQMwi7AShFMEU&iact=mrc&uact=8 • https://www.google.com/imgres?imgurl=https%3A%2F%2Fwww.simmonsandfletcher.com%2Fwp-content%2Fuploads%2F2013%2F10%2Fsiberian- 1024x681.jpg&imgrefurl=https%3A%2F%2Fwww.simmonsandfletcher.com%2Fdog-bites-attacks%2Fsiberian-husky%2F&docid=gu79vWo- 9FTgOM&tbnid=EDTEYEqIgCsf9M%3A&vet=10ahUKEwjejrGW-N7fAhXoRd8KHTwBAiwQMwifASgpMCk..i&w=1024&h=681&bih=955&biw=1920&q=husky&ved=0ahUKEwjejrGW- N7fAhXoRd8KHTwBAiwQMwifASgpMCk&iact=mrc&uact=8 • https://www.google.com/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwjby6DZ- t7fAhXtnuAKHTAVDj8QjRx6BAgBEAU&url=https%3A%2F%2Ffineartamerica.com%2Ffeatured%2Ftimber-wolf-in-snow-michael- cummings.html&psig=AOvVaw3WCHo7r6pWEpruflUEhJWS&ust=1547063430131191 • https://www.google.com/imgres?imgurl=http%3A%2F%2Fpixdaus.com%2Ffiles%2Fitems%2Fpics%2F4%2F92%2F261492_6fec0d2047da01e45439ee48aa4dd7d0_large.jpg&imgrefurl=http%3A%2F %2Fpixdaus.com%2Fwolf-running-in-the-snow-wolf-snow-running%2Fitems%2Fview%2F261492%2F&docid=xbmlvaU1Oqn-mM&tbnid=vHRg4XDHelvXCM%3A&vet=10ahUKEwj-ndDC- t7fAhWyTt8KHVBSBcwQMwiRASgsMCw..i&w=900&h=563&bih=955&biw=1920&q=wolf%20with%20snow&ved=0ahUKEwj-ndDC-t7fAhWyTt8KHVBSBcwQMwiRASgsMCw&iact=mrc&uact=8 • https://www.google.com/imgres?imgurl=https%3A%2F%2Fcache.desktopnexus.com%2Fthumbseg%2F1472%2F1472148- bigthumbnail.jpg&imgrefurl=https%3A%2F%2Fanimals.desktopnexus.com%2Fwallpaper%2F1472148%2F&docid=js6YQW-yLFpUZM&tbnid=e8VHz- IghbdanM%3A&vet=10ahUKEwiysZTV_t7fAhUjheAKHVCKBN0QMwhFKAcwBw..i&w=450&h=337&bih=955&biw=1920&q=black%20and%20white%20wolf%20with%20snow&ved=0ahUKEwiysZTV_ t7fAhUjheAKHVCKBN0QMwhFKAcwBw&iact=mrc&uact=8 • https://arxiv.org/pdf/1602.04938v1.pdf • https://christophm.github.io/interpretable-ml-book/lime.html