External validation of prognostic model of tbi
Download as PPTX, PDF1 like543 views
This document describes the external validation of the CRASH prediction model for mortality after traumatic brain injury (TBI) using data from the NIMHANS ICU in Bangalore, India. The CRASH model was validated on 150 TBI patients admitted to the ICU. Three variables from the CRASH model - pre-ICU Glasgow Coma Scale, obliteration of the third ventricle/basal cisterns on CT scan, and complications during ICU stay - were identified as significant predictors of mortality. The validated model demonstrated excellent discrimination and calibration for predicting mortality in this population. However, the study notes that prediction models may need to be customized for individual ICUs as patient characteristics and services change over time.
External validation of prognostic model of tbi
- 1. External Validation of a Prognostic Model to Predict Mortality After Traumatic Brain Injury Dhaval Shukla*, Akhil Deepika*, GS Umamaheshwar Rao#, DK Subbukrishna@ Departments of Neurosurgery*, Neuroanesthesiology#, and Biostatistics@ NIMHANS, Bangalore
- 2. Prediction Models • Statistical models that combine two or more items of patient data to predict outcome • Two requirements – clinically valid – methodologically valid • More reliable than what doctors can foretell • Influence patient management
- 3. Hierarchy of Prediction Models • Univariate Analysis • Multivariate Analysis • Logistic Regression Analysis • Discriminant Analysis • Web Based Calculator
- 4. Clinical Predictors • GCS • Motor Response • Pupillary Reaction • Ocular Movements • Blood Pressure
- 5. CT Scan Predictors • Midline Shift • Cisterns • Ventricles • Hematomas • Petechial Hemorrhages
- 6. Biochemical Predictors • Oxygen • Hemoglobin • Glucose
- 7. 102 Prediction Models of TBI • Small Sample Size • Logistic Regression • 93% High Income Countries • 11% External Validation • 19% User-friendly Perel, et al. BMC Medical Informatics and Decision Making 2006
- 9. External Validation of CRASH Prediction Model from IMPACT Dataset Perel, et al. BMJ 2008
- 10. External Validation King: Tell me about my future Soothsayer 1: Your all relatives will DIE in front of your eyes King punishes him. Soothsayer 2: You will LIVE longest King rewards him. Its only human to cross check if someone predicts bad about you
- 11. External Validation • The performance of a model on a different population (‘generalizability’ or ‘transportability’) • CRASH model not validated in middle/ low income country BMJ 2008;336:425
- 12. • Review of clinical and CT Scan data of consecutively admitted TBI patients in ICU over 6 months DATA COLLECTION • Univariable Logistic Regression Analyses (LRA) • Multivariate LRA MODEL CONSTRUCTI ON • Discrimination • Calibration MODEL PERFORMANCE • Bootstrap MethodVALIDATION Indicates how closely predicted outcomes match observed outcomes Resample from the sample data at hand for approximating sampling distribution of a statistic and bias correction Describes how well a model distinguishes between those who die from those who survive 0.90-1 is Excellent
- 13. Demographics • Total no of patients: 150 • Male : Female :: 5.5 : 1 • Age range: 1 to 85 • Mean ICU stay: 8.3±7.2 days • Mortality: 15.3% • Time to death: 7.52±4.56 days
- 14. Variable CRASH OR (95%CI) NIMHANS OR (95%CI) p Value Age 1.46 (1.39 to 1.54) 0.99 (0.95 to 1.04) GCS 1.27 (1.24 to 1.31) 2.14 (1.27 to 3.60) 0.004 Pupil Reaction Both One None 1 1.45 (1.14 to 1.86) 3.12 (2.46 to 3.97) 1 1.30 (0.3 to 43.7) 1.23 (0.4 to 32.66) Extracranial Injury 1.08 (0.91 to 1.28) CT Scan Petechial Hemorrhages 1.26 (1.07 to 1.47) 0.81 (0.16 to 4.00) Obliteration of 3rd Ventricle/ Basal Cisterns 1.99 (1.69 to 2.35) 7.32 (1.27 to 42.14) 0.026 SAH 1.33 (1.14 to 1.55) 0.98 (0.23 to 4.17) Midline Shift 1.78 (1.44 to 2.21) 0.42 (0.06 to 2.63) Non Evacuated Hematoma 1.48 (1.24 to 1.76) 0.70 (0.00 to 1.70) Complications in ICU 0.04 (0.00 to 0.29) 0.001
- 15. Model Construction • CRASH Variables • 3 variables from univariate analysis – (Pre ICU GCS, Intubation, Complication ) Result • Pre ICU GCS (P_GCS) • Obliteration of 3rd ventricle/ Basal Cistern (OB) • Complication during stay (CD)
- 16. Discrimination Variables Adjusted OR 95% Confidence Interval Lower Upper OB 3.565 1.069 11.882 P_GCS 1.819 1.242 2.662 CD 0.053 0.011 0.262
- 17. Performance of Model Area : 0.925, 95% CI : 0.877 – 0.973
- 18. Calibration Calibration: Chi2 : 4.796, Significance: 0.685
- 19. Internal Validation Predicted group * OUTCOMECrosstabulation 13 3 16 72.2% 4.0% 17.2% 5 72 77 27.8% 96.0% 82.8% 18 75 93 100.0% 100.0% 100.0% Count % within OUTCOME Count % within OUTCOME Count % within OUTCOME DEAD SURVIVED Predicted group Total DEAD SURVIVED OUTCOME Total Overall Accuracy 91.4%
- 20. Explanation • CRASH model never validated for developing countries • Only ICU patients were sampled • Many patients underwent surgery • Pre ICU GCS
- 21. Conclusions • Prediction models based on large population studies may not be valid for a selected group of patients • Each intensive care should have their own prediction models, which should be revised when services improve
Editor's Notes
- #3: For a prognostic model to be clinically useful it should fulfil two requirements: it must be clinically valid and methodologically valid
- #11: Same thing happened to me. When I ran variables of our patients in ICU in CRASH model, I go exceptionally high mortality, which did not seem true with our observation, hence I went to Dr. Subbukrishna to ask the question? Whether this model is wrong? He said, “This model is correct, probably your patients are wrong”. CRASH model gives double mortality for developing country This prompted me to validate this model for our patients
- #12: IMPACT (international mission for prognosis and clinical trial) dataset).
- #13: Resample from the sample data at hand for approximating the sampling distribution of a statistic and bias correction Discrimination: Calibration: Bootstrap method : use the data of a sample study at hand as a “surrogate population”, for the purpose of