Modern modeling techniques had limited external validity in predicting mortality from traumatic brain injury

详细信息    查看全文

• 作者：Tjeerd van der Ploeg^a ; ^b ; ^c ; ^{tvdploeg@quicknet.nl" class="auth_mail" title="E-mail the corresponding author} ; Daan Nieboer^c ; Ewout W. Steyerberg^c
• 关键词：Prediction models ; Modeling techniques ; Internal validation ; External validation ; Discrimination ; Calibration
• 刊名：Journal of Clinical Epidemiology
• 出版年：2016
• 出版时间：October 2016
• 年：2016
• 卷：78
• 期：Complete
• 页码：83-89
• 全文大小：775 K

文摘

Prediction of medical outcomes may potentially benefit from using modern statistical modeling techniques. We aimed to externally validate modeling strategies for prediction of 6-month mortality of patients suffering from traumatic brain injury (TBI) with predictor sets of increasing complexity.

Methods

We analyzed individual patient data from 15 different studies including 11,026 TBI patients. We consecutively considered a core set of predictors (age, motor score, and pupillary reactivity), an extended set with computed tomography scan characteristics, and a further extension with two laboratory measurements (glucose and hemoglobin). With each of these sets, we predicted 6-month mortality using default settings with five statistical modeling techniques: logistic regression (LR), classification and regression trees, random forests (RFs), support vector machines (SVM) and neural nets. For external validation, a model developed on one of the 15 data sets was applied to each of the 14 remaining sets. This process was repeated 15 times for a total of 630 validations. The area under the receiver operating characteristic curve (AUC) was used to assess the discriminative ability of the models.

Results

For the most complex predictor set, the LR models performed best (median validated AUC value, 0.757), followed by RF and support vector machine models (median validated AUC value, 0.735 and 0.732, respectively). With each predictor set, the classification and regression trees models showed poor performance (median validated AUC value, <0.7). The variability in performance across the studies was smallest for the RF- and LR-based models (inter quartile range for validated AUC values from 0.07 to 0.10).

Conclusion

In the area of predicting mortality from TBI, nonlinear and nonadditive effects are not pronounced enough to make modern prediction methods beneficial.