Using Clinical Treatment Data in a Machine Learning Approach for Sepsis Detection

摘要 意义：我们建议在随机对照试验（RCT）中评估HindSight的性能。 HindSight是一种新型编码软件，旨在优化脓毒症预测和检测的警报。 HindSight在前患者的电子健康记录中识别临床医生的脓毒症相关决策， 然后使用这些事件为InSight提供真阳性脓毒症病例的标记示例。在我们 回顾性的工作，我们已经表明，HindSight使洞察力能够适应现实世界的特质 通过成功减少错误和不相关的警报，无需人工监督，实现临床部署。目标 这个项目的目的是证明HindSight的回顾性成功可以成功地转化为 现场临床环境。研究问题：基于机器学习的贴标机在多大程度上可以 回顾性地学会了根据临床医生标记的脓毒症黄金自主标记脓毒症病例 标准，成功减少前瞻性随机对照试验中的错误警报？此工具是否会比 脓毒症CDS工具的设计不能自动重现脓毒症的临床识别？以前的工作： 在我们的I期研究中，HindSight在临床脓毒症评估中的AUROC分别为0.899、0.831和0.877， 治疗和发作。通过使用在线学习算法将HindSight标记的数据 在InSight预测器中，我们表明在线训练的InSight可以适应HindSight标记的数据 并且优于InSight的基线和定期重新训练版本（p < 0.05）。具体目标： 前瞻性地验证HindSight在四家不同医院的实时患者数据流上的性能 非介入性（目标1）;以及在前瞻性、介入性RCT中评价该工具的效果（目标2）。 方法：HindSight将在四家学术和社区医院的背景下进行评估。以下 任何必要的算法优化来自现场医院验证，我们将进行随机对照试验，以评估 与以下各项相比，在HindSight脓毒症标签上训练的InSight（实验组）的错误警报减少 InSight接受金标准脓毒症-3标签培训（对照组）。关注的主要结局指标将是 减少误报通过阳性预测值（PPV）证明目标1的成功完成 在95%置信区间下限满足或超过 来自先前回顾性研究的基准。符合回顾性PPV基准表明， 前瞻性CDS质量反映了回顾性CDS质量，并且足够高以减少警报疲劳， 提高临床实用性。目标2的成功取决于实现15%的错误警报相对减少 当在两个治疗组之间进行比较时（p < 0.05; Fisher精确检验）。未来方向：The 在多中心、交叉RCT中，HindSight对减少错误警报的影响的临床验证将 证明警报符合当地临床实践，并将促进新医院的商业扩张 系统.

Abstract Significance: We propose to evaluate the performance of HindSight in a randomized controlled trial (RCT). HindSight is a novel encoding software designed to optimize alerts for sepsis prediction and detection. HindSight identifies clinicians’ sepsis-related decisions in the electronic health records of former patients and then uses these events to supply InSight with labeled examples of true positive sepsis cases. In our retrospective work, we have shown that HindSight enables InSight to adapt to the idiosyncrasies of real-world clinical deployment by successfully reducing false and irrelevant alarms, without human supervision. The goal of this project is to demonstrate that the retrospective success of HindSight can be successfully translated to live clinical environments. Research Question: To what extent can a machine-learning-based labeler, which has retrospectively learned to autonomously label sepsis cases according to a clinician-labeled sepsis gold standard, successfully reduce false alerts in a prospective RCT? Will this tool perform more successfully than a sepsis CDS tool that is not designed to autonomously reproduce clinician identification of sepsis? Prior Work: In our Phase I work, HindSight achieved an AUROC of 0.899, 0.831 and 0.877 for clinician sepsis evaluation, treatment, and onset, respectively. By using an online learning algorithm to incorporate HindSight-labeled data into the InSight predictor, we showed that the online-trained InSight can adapt to the HindSight-labeled data and outperform both baseline and periodically re-trained versions of InSight (p < 0.05). Specific Aims: To prospectively validate HindSight’s performance on real-time patient data streams in four diverse hospitals non-interventionally (Aim 1); and to evaluate the effect of the tool in a prospective, interventional RCT (Aim 2). Methods: HindSight will be evaluated in the background at four academic and community hospitals. Following any necessary algorithm optimization arising from live hospital validation, we will perform an RCT to evaluate reductions in false alerts from InSight trained on HindSight sepsis labels (experimental arm), compared to InSight trained on gold standard Sepsis-3 labels (control arm). The primary outcome measure of interest will be false alert reduction. Successful completion of Aim 1 will be demonstrated by a positive predictive value (PPV) in a live clinical setting for which the lower bound of the 95% confidence interval meets or exceeds the benchmark from prior retrospective studies. Meeting the retrospective PPV benchmark indicates that prospective CDS quality reflects retrospective CDS quality, and is sufficiently high to reduce alarm fatigue and improve clinical utility. Success of Aim 2 is contingent upon achieving a 15% relative reduction in false alerts when comparing between the two treatment arms (p < 0.05; Fisher’s Exact Test). Future Directions: The clinical validation of HindSight’s impact on reducing false alerts in a multi-center, cross-ward RCT will demonstrate that alerts match local clinical practice and will promote commercial expansion to new hospital systems.