Opening The Black Box: Enhancing Machine Learning Interpretability To Optimize Clinical Response To Sudden Deterioration In COVID-19 Patients

项目摘要/摘要 高级机器学习(ML)一直被证明比专家意见更好，而且更简单 用于预测临床结果的分析。然而，很少有成功的前瞻性临床病例。 这些工具的实现。在医疗保健领域实施和采用高级ML的独特障碍 以下是(1)在现有工作流中实时运行和显示这些模型的技术挑战 以及(2)高技能提供商普遍不信任黑匣子算法。因此，承诺的 这些工具很大程度上消失在医疗保健领域。这在新冠肺炎上尤其成问题，在那里患者可以 病情迅速恶化，从看起来稳定到突然呼吸衰竭或休克，几乎没有明显的症状 警告。及早认识到这种恶化对于能够改善结果的积极干预至关重要。 Ecart是芝加哥大学过去迭代开发的预测性分析方法 十年，以确定有急性临床恶化风险的住院患者。一个简单的(基于逻辑回归的) ML模型(ECARTv2)可在AgileMD临床决策的电子健康记录中商业使用 支撑平台。ECARTv2是在一个追溯的多中心数据集中开发的，它在临床实践中的应用 在一项多中心试验中，与死亡率降低29%的相对风险相关。我们的团队最近完成了 开发和验证该模型的梯度助推机(GBM)版本(ECARTv4)，使用 100个变量，包括趋势和互动。高级ML模型的准确度明显高于 用于预测所有医院的急性临床恶化的简单ML和其他模型，在两种情况下 脓毒症患者和非脓毒症患者以及新冠肺炎患者。下一个挑战是在临床上实施它。 该项目的目标是a)升级现有的AgileMD平台，以支持以前派生的 验证eCARTv4型号并彻底改造人机界面以实现高级用户体验(UX) 这第一次提供了可解释的、图形化的洞察单个变量对 实时嵌入EHR的高级ML分析，以及b)衡量新工具对HCP的影响 效力、效率和满意度。我们假设高准确度和高精度的组合 高级ML和UX提供的可解释性也将导致对急性恶化的更早识别 作为增加的系统可用性评分(SUS)和有用性评分在治疗恶化的新冠肺炎 病人超过标准护理。

Project Summary/Abstract Advanced machine learning (ML) has consistently been shown to outperform expert opinion and more simple analytics for predicting clinical outcomes. However, there has been a paucity of successful prospective clinical implementations of such tools. The unique barriers to advanced ML implementation and adoption in healthcare are (1) the technological challenges of running and displaying these models in real-time within existing workflows and (2) a general distrust for black box algorithms among highly skilled providers. As a result, the promise of these tools is largely lost in healthcare. This is particularly problematic in COVID-19, where patients can deteriorate rapidly, from appearing stable to suddenly being in respiratory failure or shock with little obvious warning. Early recognition of this deterioration is vital to proactive interventions, which can improve outcomes. eCART is a predictive analytic that has been developed iteratively at the University of Chicago over the past decade to identify hospitalized patients at risk for acute clinical deterioration. A simple (logistic regression based) ML model (eCARTv2) is commercially available within electronic health records on AgileMD’s clinical decision support platform. eCARTv2 was developed in a retrospective multicenter dataset and its use in clinical practice was associated with a 29% relative risk reduction in mortality in a multicenter trial. Our team recently completed development and validation of a gradient boosted machine (GBM) version of the model (eCARTv4), using nearly 100 variables, including trends and interactions. The advanced ML model was significantly more accurate than the simple ML and other models for predicting acute clinical deterioration across all hospital settings, in both septic and non-septic patients as well as in COVID-19 patients. The next challenge is clinically implementing it. The goals of this project are to a) upgrade the existing AgileMD platform to support the previously derived and validated eCARTv4 model and overhaul the human-machine interface for an advanced user experience (UX) that provides, for the first time, interpretable, graphical insight into the contribution of individual variables to a real-time EHR-embedded advanced ML analytic, and b) measure the impact of the new tool on HCP effectiveness, efficiency and satisfaction. We hypothesize that the combination of high accuracy and interpretability afforded by the advanced ML and UX will result in earlier recognition of acute deterioration as well as increased System Usability Scores (SUS) and usefulness scores in the treatment of deteriorating COVID-19 patients over standard care.