Adding Social Determinant Data Changes Children’s Hospitals’ Readmissions Performance
1 like460 views
Adding social determinant data to risk adjustment models for pediatric readmissions led to minimal changes in model performance at the discharge level, but resulted in changes to hospital performance rankings. Specifically: - Adding social determinant variables from electronic health records and zip codes to existing clinical risk adjustment models did not meaningfully improve the accuracy or fit of models predicting individual readmissions. - However, accounting for social determinants did change some hospitals' risk-adjusted readmission rates and performance deciles compared to peers. This suggests social determinants may influence hospital performance evaluations and penalties if unadjusted. - Including social determinants in readmissions modeling more fully captures factors influencing readmissions and provides a more accurate assessment of hospital quality.
Adding Social Determinant Data Changes Children’s Hospitals’ Readmissions Performance
- 1. Adding Social Determinant Data Changes Children’s Hospitals’ Readmissions Performance Children’s Hospital Colorado Marion R. Sills, MD, MPH Matt Hall, PhD Gretchen J. Cutler, PhD, MPH Jeffrey D. Colvin, MD, JD Laura M. Gottlieb, MD, MPH Michelle L. Macy, MD, MS Jessica L. Bettenhausen, MD Rustin B. Morse, MD Evan S. Fieldston, MD, MBA, MSHP Jean L. Raphael, MD, MPH Katherine A. Auger, MD, MSc Samir S. Shah, MD, MSCE
- 2. MARION SILLS has documented no financial relationships to disclose or Conflicts of Interest (COIs) to resolve
- 3. Background: Readmissions-related policy • Affordable Care Act
- 4. Background: Readmissions-related policy • Affordable Care Act: Centers for Medicare and Medicaid Services (CMS) must improve quality, as reflected by readmissions Quality of inpatient care Readmissions
- 5. Background: Readmissions-related policy • CMS implemented readmissions penalties nationwide (adults) • Several state Medicaid agencies implemented readmissions penalties (currently 8 states) 5 Readmissions Readmission penalty Quality of inpatient care
- 6. Background: Risk adjustment • Although readmission penalties were designed to incentivize high quality inpatient care… Readmissions Readmission penalty Quality of inpatient care
- 7. Background: Risk adjustment • … pediatric studies: other factors—beyond the hospital’s influence-- are associated with readmission Readmissions Readmission penalty Medical factors Social determinants of health (SDH) Quality of inpatient care
- 8. Background: Risk adjustment • Prior pediatric studies: medical factors and social determinants are associated with readmission penalties 8 Readmissions Readmission penalty Medical factors Social determinants of health Quality of inpatient care Sills MR, Hall M, Colvin JD, et al. Association of Social Determinants With Children's Hospitals' Preventable Readmissions Performance. JAMA Pediatr. 2016 Apr;170(4):350-8.
- 9. Background: Risk adjustment • State Medicaid policies: allow risk-adjustment for some medical factors; not for SDH 9 Readmissions Readmission penalty Medical factors Social determinants of health Quality of inpatient care
- 10. Background: Risk adjustment • No prior studies measured impact of social determinants on both readmissions and readmission penalties in children Medical factors Social determinants of health Quality of inpatient care Readmissions Readmission penalty
- 11. Background: Risk adjustment • No prior studies measured impact of social determinants on both readmissions and readmission penalties in children Discharge-level outcome: Readmissions Hospital-level outcome: Readmission penalty Medical factors Social determinants of health Quality of inpatient care
- 12. Objectives • To determine whether risk adjustment for SDH in models of short term (30-day) all-condition readmissions Discharge-level outcome: Readmissions Hospital-level outcome: Readmission penalty • changes hospital-level performance • improves fit and accuracy of discharge-level models Medical factors Social determinants of health
- 13. Methods • Retrospective cohort study • Data source: 47 children’s hospitals in the Pediatric Health Information System (PHIS), including non-freestanding • Study period: calendar year 2014
- 14. Methods: Readmissions Measure • Pediatric All Condition Readmissions (PACR) • Selected because its specifications are publicly available and NQF-endorsed • PACR cohort definition = study cohort definition • Includes acute care hospitalizations for patients <18 years • Excludes hospitalizations • of newborns • for obstetrics • for psychiatry • for a planned procedure 14
- 15. Methods: Readmissions Measure • Pediatric All Condition Readmissions (PACR) • PACR risk-adjusts for • Chronic conditions: uses Chronic Condition Indicators (CCI) • from Healthcare Cost and Utilization Project (HCUP) • CCI categorizes chronic conditions into 17 body systems • PACR measures CCI as 17 indicator variables and a count of number of body systems involved • 2 SDH variables: age and sex 1515
- 16. Methods: Outcome measure • Discharge-level outcome: 30-day PACR 16 Discharge-level outcome: Readmissions Hospital-level outcome: Readmission penalty • Hospital-level outcome: change in hospital decile rank on 30-day PACR Medical factors Social determinants of health
- 17. Methods: Models • Sequential models, added risk-adjustment variables: • Model 1: CCI only 17
- 18. Methods: Models • Sequential models, added risk-adjustment variables: • Model 1: CCI only • Model 2: Model 1 + age + sex (= the PACR risk adjustment model) 18 Social determinants of health (SDH)
- 19. Methods: Models • Sequential models, added risk-adjustment variables: • Model 1: CCI only • Model 2: Model 1 + age + sex (= the PACR risk adjustment model) • Model 3: Model 2 + electronic health record (EHR)-derived SDH • race • ethnicity • payer 19 Social determinants of health (SDH)
- 20. Methods: Models • Sequential models, added risk-adjustment variables: • Model 1: CCI only • Model 2: Model 1 + age + sex (= the PACR risk adjustment model) • Model 3: Model 2 + EHR-derived SDH • Model 4: Model 3 + zip-code-linked SDH • Median household income • Proportion of housing units vacant • Proportion of households with children that are single-parent • Proportion of families below poverty level • Unemployment rate • Proportion of adults with less than a high school diploma or equivalent 20 Social determinants of health (SDH)
- 21. Methods: Discharge-Level Analysis • Models: generalized linear mixed effects models • To assess model accuracy: • c-statistic for each model • significance of the improvement in c-statistic between sequential models • To test improvement in model fit: Likelihood Ratio test • Performed for each pair of successive models • Compares the goodness-of-fit of two models 21 Discharge-level outcome: Readmissions
- 22. Methods: Hospital-Level Analyses • Calculated hospital-level unadjusted and adjusted readmission rates for the 4 models • Calculated change in hospital-level rank-decile 22 Hospital-level outcome: Readmission penalty
- 23. 23 Risk Adjustment Factors: Known determinants of pediatric readmissions Study OutcomesUnmeasured Risk-adjustment factors in PACR • Model 1: Chronic conditions • Model 2: Model 1 + age + sex Additional SDH adjustment • Model 3: Model 2 + EHR-derived SDH • Model 4: Model 3 + zip-code-linked SDH Process measure: Quality of hospital’s inpatient care Discharge-level outcome: Readmission rate (change in model accuracy and fit) Hospital-level outcome: Readmission penalty (readmission rank decile change) Methods: Summary of Study Design
- 24. 24 Results: Study population characteristics EHR-derived SDH variables • 30% infants • 54% male • 58% white • 21% Hispanic • 59% Medicaid-insured • 39% had conditions involving >2 of CCI’s 17 body systems Zip-linked SDH variables • 8% families <poverty level • median household income: $39,000 • 6% housing units vacant • 10% unemployment rate • 14% adults with < high school diploma • 27% households with children that are single-parent • 458,686 index discharges
- 25. 25 Results: PACR Rates Q1: 5.3% Discharge-level readmission rate Hospital-level readmission rate Q3: 7.0% Median: 6.1% Mean: 6.1% Max: 8.6%Min: 4.2% IQR
- 26. 26 Results: Discharge-level models: Model 4 EHR-derived SDH variables • Age < 1 year • Female sex • 1 of the 17 CCI body systems (respiratory) • Having a larger number of CCI body systems involved • Medicaid insurance • White race Zip-linked SDH variables • Higher median household income (unadjusted change: $64) • Higher percentage of single parent households (unadjusted change: 0.1%) • Factors increasing risk of 30-day PACR (all p <0.001)
- 27. 27 Model C-statistic Rise in c- statistic (p value) Likelihood Ratio test Preferred model (p value) Model 1: Readmissions adjusted for CCI’s 0.704 <0.001 Model 2 (<0.001)Model 2: Model 1 plus age and sex (the full PACR) 0.707 <0.001 Model 3 (<0.001)Model 3: Model 2 plus EHR- derived SDH 0.708 0.385 Model 4 (0.011)Model 4: Model 3 plus zip-linked SDH 0.708 Results: Discharge-level Models: Accuracy Statistics C-statistic values: reasonable
- 28. 28 Model C-statistic Rise in c- statistic (p value) Likelihood Ratio test Preferred model (p value) Model 1: Readmissions adjusted for CCI’s 0.704 <0.001 Model 2 (<0.001)Model 2: Model 1 plus age and sex (the full PACR) 0.707 <0.001 Model 3 (<0.001)Model 3: Model 2 plus EHR- derived SDH 0.708 0.385 Model 4 (0.011)Model 4: Model 3 plus zip-linked SDH 0.708 Results: Discharge-level Models: Accuracy Statistics PACR PACR PACR + SDHSocial determinants of health
- 29. 29 Model C-statistic Rise in c- statistic (p value) Likelihood Ratio test Preferred model (p value) Model 1: Readmissions adjusted for CCI’s 0.704 <0.001 Model 2 (<0.001)Model 2: Model 1 plus age and sex (the full PACR) 0.707 <0.001 Model 3 (<0.001)Model 3: Model 2 plus EHR- derived SDH 0.708 0.385 Model 4 (0.011)Model 4: Model 3 plus zip-linked SDH 0.708 Results: Discharge-level Models: Accuracy Statistics Minimal change
- 30. 30 Model C-statistic Rise in c- statistic (p value) Likelihood Ratio test Preferred model (p value) Model 1: Readmissions adjusted for CCI’s 0.704 <0.001 Model 2 (<0.001)Model 2: Model 1 plus age and sex (the full PACR) 0.707 <0.001 Model 3 (<0.001)Model 3: Model 2 plus EHR- derived SDH 0.708 0.385 Model 4 (0.011)Model 4: Model 3 plus zip-linked SDH 0.708 Results: Discharge-level Models: Goodness of Fit Statistics
- 31. 31 Model C-statistic Rise in c- statistic (p value) Likelihood Ratio test Preferred model (p value) Model 1: Readmissions adjusted for CCI’s 0.704 <0.001 Model 2 (<0.001)Model 2: Model 1 plus age and sex (the full PACR) 0.707 <0.001 Model 3 (<0.001)Model 3: Model 2 plus EHR- derived SDH 0.708 0.385 Model 4 (0.011)Model 4: Model 3 plus zip-linked SDH 0.708 Results: Discharge-level Models: Summary Summary of discharge-level findings: adding SDH led to minimal changes in model performance
- 32. 32 Results: Hospital Rank Decile by Readmissions Rate • Lower rank decile: lower readmission rate (blue) • Higher decile rank: higher readmission rate (red)
- 33. 33 Results: Hospital Rank Decile by Readmissions Rate
- 34. 34 Results: Hospital Rank Decile by Readmissions Rate 8 hospitals: higher decile (worse) 9 hospitals: lower decile (better) 17 hospitals (36%) changed decile rank
- 35. Discussion: Discharge-level Analysis • Consistent with prior studies, we found an association between SDH with readmissions • However, addition of SDH risk-adjustment: minimal improvement in the fit and accuracy of prediction models for PACR • Model performance fair: readmissions hard to predict! 35 Model C-statistic Model 2: Model 1 plus age and sex (the PACR) 0.707 Model 4: Model 3 plus zip-linked SDH 0.708
- 36. Discussion: Discharge-level Analysis • Why such small improvement in model performance? • High risk SDH (e.g. low income) associated with chronic medical complexity • Thus, CCI-adjustment (Model 1) also adjusted for SDH Readmissions Medical factors: CCI Social determinants of health (SDH)
- 37. Discussion: Hospital-level Analysis • SDH risk adjustment: substantial changes in rank decile of hospitals’ readmission performance • Consistent with our prior hospital-level analysis: SDH-adjustment changes hospital-level pediatric readmission penalty status • Consequences for hospitals could be substantial 37
- 38. Limitations • PHIS dataset represents only children’s hospitals: • May limit variation between hospitals • May limit generalizability to children hospitalized in other settings • Use of decile rankings can distort the effect of outcome change • the number of rank changes is related to the number of ranks
- 39. Conclusions • SDH risk adjustment using variables common in or linkable to electronic data • Substantial impact on readmissions performance at hospital-level and thus can impact penalties • Small impact on readmission model performance at discharge-level • Findings support inclusion of SDH variables in risk- adjustment of hospital-level pay for performance measures 39 Discharge-level outcome: Readmissions Hospital-level outcome: Readmission penalty Social determinants of health (SDH)
- 40. Matt Hall, PhD Gretchen J. Cutler, PhD, MPH Jeffrey D. Colvin, MD, JD Laura M. Gottlieb, MD, MPH Michelle L. Macy, MD, MS Jessica L. Bettenhausen, MD Rustin B. Morse, MD Evan S. Fieldston, MD, MBA, MSHP Jean L. Raphael, MD, MPH Katherine A. Auger, MD, MSc Samir S. Shah, MD, MSCE Acknowledgments Citation Co-authors Sills MR et al. Adding Social Determinant Data Changes Children’s Hospitals’ Readmissions Performance. J Pediatrics 2017 Marion.Sills@ucdenver.edu
- 41. Discharge-level Analysis • How do we better predict readmission? • Include other variables • measures of SDH not available in our dataset • other measures: e.g., prior utilization patterns, access to care and health literacy 41
- 42. 42
- 43. 43
- 44. 44