Using Clinical Treatment Data in a Machine Learning Approach for Sepsis Detection

在机器学习方法中使用临床治疗数据进行脓毒症检测

• 批准号: 10258043
• 负责人: Jana Hoffman
• 金额: $ 199.96万
• 依托单位: DASCENA, INC.
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-06-15 至 2022-06-15
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10258043
• 关键词: Adult; Algorithms; Behavior; Benchmarking; Cessation of life; Characteristics; Charge Nurses; Client; Clinical; Clinical Decision Support Systems; Clinical Treatment; Community Hospitals; Computer software; Confidence Intervals; Data; Detection; E-learning; Early Diagnosis; Early Intervention; Electronic Health Record; Emergency Department patient; Enrollment; Environment; Evaluation; Event; Fatigue; Future; Goals; Gold; Healthcare Systems; Hospitals; Human; Immune response; Inpatients; Institution; Intervention; Judgment; Label; Learning; Location; Machine Learning; Methods; Modeling; Modification; Monitor; Nature; Outcome Measure; Patient-Focused Outcomes; Patients; Pattern; Performance; Phase; Physicians; Positioning Attribute; Predictive Value; Randomized Controlled Trials; Research; Retrospective Studies; Risk; Sensitivity and Specificity; Sepsis; Signal Transduction; Site; Software Design; Specificity; Supervision; Symptoms; System; Systemic Inflammatory Response Syndrome; Testing; Time; Training; Training Activity; Translating; Treatment outcome; Validation; Work; arm; base; clinical decision support; clinical practice; clinically relevant; commercialization; cost; data streams; experience; experimental arm; falls; high risk; improved; insight; interest; learning algorithm; meetings; novel; primary endpoint; primary outcome; prospective; research clinical testing; response; septic; success; support tools; tool; treatment arm; ward

Abstract Significance: We propose to evaluate the performance of HindSight in a randomized controlled trial (RCT). HindSight is a novel encoding software designed to optimize alerts for sepsis prediction and detection. HindSight identifies clinicians’ sepsis-related decisions in the electronic health records of former patients and then uses these events to supply InSight with labeled examples of true positive sepsis cases. In our retrospective work, we have shown that HindSight enables InSight to adapt to the idiosyncrasies of real-world clinical deployment by successfully reducing false and irrelevant alarms, without human supervision. The goal of this project is to demonstrate that the retrospective success of HindSight can be successfully translated to live clinical environments. Research Question: To what extent can a machine-learning-based labeler, which has retrospectively learned to autonomously label sepsis cases according to a clinician-labeled sepsis gold standard, successfully reduce false alerts in a prospective RCT? Will this tool perform more successfully than a sepsis CDS tool that is not designed to autonomously reproduce clinician identification of sepsis? Prior Work: In our Phase I work, HindSight achieved an AUROC of 0.899, 0.831 and 0.877 for clinician sepsis evaluation, treatment, and onset, respectively. By using an online learning algorithm to incorporate HindSight-labeled data into the InSight predictor, we showed that the online-trained InSight can adapt to the HindSight-labeled data and outperform both baseline and periodically re-trained versions of InSight (p < 0.05). Specific Aims: To prospectively validate HindSight’s performance on real-time patient data streams in four diverse hospitals non-interventionally (Aim 1); and to evaluate the effect of the tool in a prospective, interventional RCT (Aim 2). Methods: HindSight will be evaluated in the background at four academic and community hospitals. Following any necessary algorithm optimization arising from live hospital validation, we will perform an RCT to evaluate reductions in false alerts from InSight trained on HindSight sepsis labels (experimental arm), compared to InSight trained on gold standard Sepsis-3 labels (control arm). The primary outcome measure of interest will be false alert reduction. Successful completion of Aim 1 will be demonstrated by a positive predictive value (PPV) in a live clinical setting for which the lower bound of the 95% confidence interval meets or exceeds the benchmark from prior retrospective studies. Meeting the retrospective PPV benchmark indicates that prospective CDS quality reflects retrospective CDS quality, and is sufficiently high to reduce alarm fatigue and improve clinical utility. Success of Aim 2 is contingent upon achieving a 15% relative reduction in false alerts when comparing between the two treatment arms (p < 0.05; Fisher’s Exact Test). Future Directions: The clinical validation of HindSight’s impact on reducing false alerts in a multi-center, cross-ward RCT will demonstrate that alerts match local clinical practice and will promote commercial expansion to new hospital systems.

摘要 意义：我们建议在随机对照试验(RCT)中评估后见之明的表现。 后见之明是一款新型编码软件，旨在优化脓毒症预测和检测的警报。 后见之明在以前患者的电子健康记录中识别临床医生与脓毒症相关的决定 然后使用这些事件提供真实阳性脓毒症病例的标签示例。在我们的 回顾工作，我们已经表明，事后诸葛亮使洞察力能够适应现实世界的特质 通过在没有人工监督的情况下成功减少错误和不相关的警报，实现临床部署。目标是 这个项目的目的是证明事后回顾的成功可以成功地转化为 活的临床环境。研究问题：基于机器学习的标签器在多大程度上可以 已学会根据临床医生标记的脓毒症金条自动标记脓毒症病例 标准，成功地减少了未来RCT中的错误警报？此工具是否会比 脓毒症CDS工具不是设计来自主复制临床医生对脓毒症的识别？之前的工作： 在我们的第一阶段工作中，事后看来，临床医生败血症评估的AUROC分别为0.899、0.831和0.877， 分别为治疗和发病。通过使用在线学习算法合并后见之明标记的数据 在洞察力预测中，我们发现在线训练的洞察力能够适应后见之明的数据 并且表现优于Insight的基准版本和定期重新培训的版本(p&lt；0.05)。具体目标： 前瞻性验证后见之明在四家不同医院的实时患者数据流上的性能 非干预性(目标1)；以及评估该工具在前瞻性干预性随机对照试验(目标2)中的效果。 方法：后见之明将在四家学术和社区医院的背景下进行评估。跟随 医院现场验证产生的任何必要的算法优化，我们将执行随机对照试验进行评估 减少事后对败血症标签(实验组)进行洞察训练的错误警报，与 对金标Sepsis-3标签(控制臂)进行洞察力培训。利益的主要结果衡量标准将是 错误警报减少。目标1的成功完成将通过阳性预测值(PPV)来证明 在95%可信区间的下限达到或超过 基准来自以前的回溯性研究。达到回溯性PPV基准表明 预期CDS质量反映回溯性CDS质量，并且足够高以减少警报疲劳和 提高临床实用性。目标2的成功取决于实现相对减少15%的错误警报 当比较两个治疗臂时(p&lt；0.05；Fisher‘s Exact Test)。未来发展方向： 在多中心、跨病房的随机对照试验中，后见之明对减少错误警报的影响的临床验证将 证明警报符合当地临床实践，并将促进商业扩展到新医院 系统。