Dynamic and explainable machine learning prediction of mortality in patients in the intensive care unit: a retrospective study of high-frequency data in electronic patient records
Dynamic and explainable machine learning prediction of mortality in patients in the intensive care unit: a retrospective study of high-frequency data in electronic patient records
复制标题
DOI：
10.1016/s2589-7500(20)30018-2
复制
发表时间：
2020-04-01
影响因子：
30.8
通讯作者：
Perner, Anders
中科院分区：
文献类型：
--
作者：
Thorsen-Meyer, Hans-Christian;Nielsen, Annelaura B.;Perner, Anders
背景：针对重症监护病房(ICU)的患者，已经开发了许多死亡率预测模型；大多数模型基于ICU入院时的数据。我们调查了使用时间序列数据分析的机器学习方法是否通过提供对90天死亡率的实时预测来改善ICU患者的死亡率预测。此外，我们研究了通过量化和可视化在不同时间点驱动预测的特征，这种动态模型可以在多大程度上被解释。方法基于简化的急性生理学评分(SAPS)III变量，我们基于2011年至2016年在丹麦首都地区四个ICU住院的患者的纵向数据训练了一个机器学习模型。我们纳入了所有年龄在16岁以上、在ICU停留时间超过1小时、拥有丹麦公民登记号以获得90天随访的患者。我们利用了来自电子健康记录和丹麦国家患者登记处的静态数据和生理时间序列数据。递归神经网络以1h的时间分辨率进行训练。该模型使用20%的训练数据集使用坚持法进行内部验证，并使用丹麦第五家医院以前未见过的数据进行外部验证。以Matthews相关系数(MCC)和受试者工作特征曲线下面积(AUROC)为度量指标，采用带替换的1000个样本的Bootstrapping方法构建95%的CIS。将Shapley加性解释算法应用到预测模型中，以获得对驱动患者特定预测的特征的解释，并分析模型中44个特征的贡献，并与原始SAPS III模型中的变量进行比较。总体而言，90天的死亡率为33.1%(3802名患者)。深度学习模型在坚持测试数据集上显示出随着ICU停留时间进程的改善的预测性能：入院时MCC 0.29(95%可信区间0.25-0.33)和AUROC 0.73(0.71-0.74)，24小时后0.43(0.40-0.47)和0.82(0.80-0.84)，72小时后0.50(0.46-0.53)和0.85(0.84-0.87)，出院时为0.57(0.54~0.60)和0.88(0.87~0.89)。该模型具有良好的定标性能。这些结果在5827名患者6748名入院的外部验证队列中得到了验证：入院时MCC 0.29(95%CI 0.27-0.32)和AUROC 0.75(0.73-0.76)，24 h后0.41(0.39-0.44)和0.80(0.79-0.81)，72 h后0.46(0.43-0.48)和0.82(0.81-0.83)，出院时分别为0.47(0.44~0.49)和0.83(0.82~0.84)。说明ICU期间抽样间隔1小时可提高对90天死亡率的预测。动态风险预测也可以解释为单个患者，可视化在任何时间点对预测有贡献的特征。这一解释允许临床医生确定在当前患者状态和护理中是否存在潜在可操作的元素，从而使该模型适合作为临床工具进行进一步验证。版权所有(C)2020作者(S)。爱思唯尔有限公司出版。
Background Many mortality prediction models have been developed for patients in intensive care units (ICUs); most are based on data available at ICU admission. We investigated whether machine learning methods using analyses of time-series data improved mortality prognostication for patients in the ICU by providing real-time predictions of 90-day mortality. In addition, we examined to what extent such a dynamic model could be made interpretable by quantifying and visualising the features that drive the predictions at different timepoints.Methods Based on the Simplified Acute Physiology Score (SAPS) III variables, we trained a machine learning model on longitudinal data from patients admitted to four ICUs in the Capital Region, Denmark, between 2011 and 2016. We included all patients older than 16 years of age, with an ICU stay lasting more than 1 h, and who had a Danish civil registration number to enable 90-day follow-up. We leveraged static data and physiological time-series data from electronic health records and the Danish National Patient Registry. A recurrent neural network was trained with a temporal resolution of 1 h. The model was internally validated using the holdout method with 20% of the training dataset and externally validated using previously unseen data from a fifth hospital in Denmark. Its performance was assessed with the Matthews correlation coefficient (MCC) and area under the receiver operating characteristic curve (AUROC) as metrics, using bootstrapping with 1000 samples with replacement to construct 95% CIs. A Shapley additive explanations algorithm was applied to the prediction model to obtain explanations of the features that drive patient-specific predictions, and the contributions of each of the 44 features in the model were analysed and compared with the variables in the original SAPS III model.Findings From a dataset containing 15 615 ICU admissions of 12 616 patients, we included 14 190 admissions of 11 492 patients in our analysis. Overall, 90-day mortality was 33.1% (3802 patients). The deep learning model showed a predictive performance on the holdout testing dataset that improved over the timecourse of an ICU stay: MCC 0.29 (95% CI 0.25-0.33) and AUROC 0.73 (0.71-0.74) at admission, 0.43 (0.40-0.47) and 0.82 (0.80-0.84) after 24 h, 0.50 (0.46-0.53) and 0.85 (0.84-0.87) after 72 h, and 0.57 (0.54-0.60) and 0.88 (0.87-0.89) at the time of discharge. The model exhibited good calibration properties. These results were validated in an external validation cohort of 5827 patients with 6748 admissions: MCC 0.29 (95% CI 0.27-0.32) and AUROC 0.75 (0.73-0.76) at admission, 0.41 (0.39-0.44) and 0.80 (0.79-0.81) after 24 h, 0.46 (0.43-0.48) and 0.82 (0.81-0.83) after 72 h, and 0.47 (0.44-0.49) and 0.83 (0.82-0.84) at the time of discharge.Interpretation The prediction of 90-day mortality improved with 1-h sampling intervals during the ICU stay. The dynamic risk prediction can also be explained for an individual patient, visualising the features contributing to the prediction at any point in time. This explanation allows the clinician to determine whether there are elements in the current patient state and care that are potentially actionable, thus making the model suitable for further validation as a clinical tool. Copyright (C) 2020 The Author(s). Published by Elsevier Ltd.