External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients
External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients
复制标题
DOI：
10.1001/jamainternmed.2021.2626
复制
发表时间：
2021-06-21
影响因子：
39
通讯作者：
Singh, Karandeep
中科院分区：
文献类型：
--
作者：
Wong, Andrew;Otles, Erkin;Singh, Karandeep
Epic Sepsis Model（ESM）是一种专有的脓毒症预测模型，已在数百家美国医院实施。ESM识别脓毒症患者的能力尚未得到充分评估，尽管广泛使用。目的外部验证ESM在预测脓毒症和评估其潜在的临床价值相比，通常care.DESIGN，设置，和参与者这项回顾性队列研究是在27697例年龄18岁或以上的患者中进行的，密歇根大学安阿伯的学术卫生系统，2018年12月6日至2019年10月20日期间有38455例住院。暴露ESM评分，每15分钟计算一次。主要结局和指标败血症，由（1）疾病控制和预防中心监测标准和（2）疾病和相关健康问题国际统计分类的复合物定义，第10次修订诊断代码伴随2个全身炎症反应综合征标准和1个器官功能障碍标准，彼此在6小时内。使用住院水平的受试者工作特征曲线下面积评估模型区分度，预测范围为4、8、12和24小时。用校准图评价模型校准。通过评价ESM评分与当代临床实践（基于及时给予抗生素）相比的额外获益，评估了与ESM相关的潜在临床获益。警报疲劳进行了评估，通过比较不同的警报strategies.RESULTS的临床价值，我们确定了27 697例患者有38 455住院（21904名妇女[57%];中位年龄，56岁[四分位数范围，35-69岁]）满足纳入标准，其中败血症发生在2552（7%）。ESM的住院水平受试者工作特征曲线下面积为0.63（95% CI，0.62-0.64）。ESM确定了2552例败血症患者中的183例（7%）没有及时接受抗生素治疗，突出了ESM与当代临床实践相比的低敏感性。ESM也没有确定1709例败血症患者（67%），尽管生成警报的ESM评分为6或更高的所有38455住院患者（18%），从而创造了一个大的负担警报fatigue.Conclusions和相关性这一外部验证队列研究表明，ESM有差的歧视和校准在预测败血症的发作。尽管ESM表现不佳，但其广泛采用引发了对国家层面脓毒症管理的根本担忧。
IMPORTANCE The Epic Sepsis Model (ESM), a proprietary sepsis prediction model, is implemented at hundreds of US hospitals. The ESM's ability to identify patients with sepsis has not been adequately evaluated despite widespread use.OBJECTIVE To externally validate the ESM in the prediction of sepsis and evaluate its potential clinical value compared with usual care.DESIGN, SETTING, AND PARTICIPANTS This retrospective cohort study was conducted among 27 697 patients aged 18 years or older admitted to Michigan Medicine, the academic health system of the University of Michigan, Ann Arbor, with 38 455 hospitalizations between December 6, 2018, and October 20, 2019.EXPOSURE The ESM score, calculated every 15 minutes.MAIN OUTCOMES AND MEASURES Sepsis, as defined by a composite of (1) the Centers for Disease Control and Prevention surveillance criteria and (2) International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnostic codes accompanied by 2 systemic inflammatory response syndrome criteria and 1 organ dysfunction criterion within 6 hours of one another. Model discrimination was assessed using the area under the receiver operating characteristic curve at the hospitalization level and with prediction horizons of 4, 8, 12, and 24 hours. Model calibration was evaluated with calibration plots. The potential clinical benefit associated with the ESM was assessed by evaluating the added benefit of the ESM score compared with contemporary clinical practice (based on timely administration of antibiotics). Alert fatigue was evaluated by comparing the clinical value of different alerting strategies.RESULTS We identified 27 697 patients who had 38 455 hospitalizations (21904 women [57%]; median age, 56 years [interquartile range, 35-69 years]) meeting indusion criteria, of whom sepsis occurred in 2552 (7%). The ESM had a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% CI, 0.62-0.64). The ESM identified 183 of 2552 patients with sepsis (7%) who did not receive timely administration of antibiotics, highlighting the low sensitivity of the ESM in comparison with contemporary clinical practice. The ESM also did not identify 1709 patients with sepsis (67%) despite generating alerts for an ESM score of 6 or higher for 6971 of all 38 455 hospitalized patients (18%), thus creating a large burden of alert fatigue.CONCLUSIONS AND RELEVANCE This external validation cohort study suggests that the ESM has poor discrimination and calibration in predicting the onset of sepsis. The widespread adoption of the ESM despite its poor performance raises fundamental concerns about sepsis management on a national level.