Engaging Multidisciplinary Health System Stakeholders to Create a Process for Implementing Machine-Learning Enabled Clinical Decision Support

项目摘要/摘要 用于临床决策支持(CDS)的黑盒机器学习(ML)模型的激增引发了 对CDS的可解释性、可操作性和总体可用性的担忧，使明确的 使包括开发人员和用户在内的各种利益相关者参与实施规划的过程。 我们的长期目标是将指导卫生系统规划、监测和评估的程序正规化。 CDS的实施。R21的总体目标是开发和评估可推广的策略 在CDS探索阶段将多学科利益相关者聚集在一起，以确定 促进者和在其背景下实施的障碍。在此过程中，我们将采用参与制 动态(PSD)建模作为评估和规划实施的多组件策略 利益相关者在实施的探索阶段，当决策发生时，以一种 支持ML的CDS可以随着时间的推移而持续。因此，我们将重点关注上游实施 支持ML的CDS的可接受性、适当性和可行性的结果。这个项目的基本原理是 尽早让不同利益攸关方参与实施规划的过程将澄清承诺 通过揭示可接受性、可行性和适当性来实现和采用的潜力。为 在这个项目中，我们将关注一组特定的启用ML的CDS：早期预警分数(EWS)，用于 识别失代偿期的患者。我们计划通过追求两个具体目标来实现我们的总体目标：1. 让参与EWS实施的多学科利益相关者(用户、开发人员、实施者、 所有者)从两个系统不同的采用环境中共同定义共同障碍，并由推动者提供关键 使用小组模型构建CDS的可接受性、适当性和可行性的实施结果 来自系统动力学领域的脚本和2.通过测量以下方面的变化来评估PSD过程 承诺采用CDS(使用可接受性、适当性和可行性的衡量标准)，引出 反馈和估计干预努力。我们将通过一系列的群体建模会议从 在不同环境中使用过CDS的利益相关者，其中警报因目标用户、时间和频率而异 在其他因素中。我们将使用系统动力学建模领域中定义良好的脚本来 促进小组讨论，以形成关于ML使能CDS反应问题的共同理论 (目标1)。由于任何战略的实施都需要适应，我们将评估私营部门司进程(AIM 2)提炼和准备在其他地方使用。这一贡献是重要的，因为EWSS被广泛使用 包括学术医院和社区医院。通过使用群组建模，这一贡献是创新的 支持ML的CDS实现问题的技术，创建了方法论和实质性的 调查结果。未来的R01将前瞻性地评估在多个用例设置中使用此流程的好处 同时继续为下游采用建立要素的动态系统模型。

PROJECT SUMMARY/ABSTRACT The proliferation of “black box” Machine Learning (ML) models for Clinical Decision Support (CDS) has raised concerns regarding CDS interpretability, actionability and overall usability, rendering a critical need for a clear process that engages various stakeholders including both developers and users in implementation planning. Our long-term goal is to formalize a process to guide health systems in planning, monitoring and evaluating CDS implementation. The overall objective for this R21 is to develop and evaluate a generalizable strategy to bring multidisciplinary stakeholders together during the CDS exploration phase to identify facilitators and barriers to implementation in their contexts. In doing so, we will use Participatory System Dynamics (PSD) modeling as a multi-component strategy to evaluate and plan implementation with stakeholders during the exploration phase of implementation, when decision-making occurs, in a way where ML-enabled CDS can be sustained over time. As such, we will focus on the upstream implementation outcomes of acceptability, appropriateness, and feasibility of ML-enabled CDS. The rationale for this project is that a process that engages diverse stakeholders in implementation planning early on will clarify commitment to implementation and potential for adoption by revealing acceptability, feasibility, and appropriateness. For this project we will focus on one particular set of ML-enabled CDS: Early Warning Scores (EWSs), used to identify decompensating patients. We plan to accomplish our overall objective by pursuing two specific aims: 1. Engage multidisciplinary stakeholders involved in EWS implementation (users, developers, implementers, owners) from two systematically varying adoption contexts to co-define common barriers and facilitators to key implementation outcomes of CDS acceptability, appropriateness, and feasibility using group model building scripts from the field of system dynamics and 2. Evaluate the PSD process by measuring change in commitment to adopt CDS (using measures of acceptability, appropriateness, and feasibility), eliciting feedback, and estimating intervention effort. We will obtain data via a series of group modeling sessions from stakeholders who have used CDS in different contexts, where alerts vary by target user, time, and frequency among other factors. We will employ well-defined scripts from the field of System Dynamics modeling to facilitate group discussion toward developing a shared theory about the problem of ML-enabled CDS response (Aim 1). Because implementation of any strategy requires adaptation, we will evaluate the PSD process (Aim 2) to refine and prepare for use elsewhere. This contribution is significant because EWSs are widely used across both academic and community hospitals. This contribution is innovative by using group modeling techniques for the problem of ML-enabled CDS implementation, creating both methodological and substantive findings. A future R01 will prospectively assess benefits of using this process in multiple use case settings while continuing to build out the dynamic systems model of factors for downstream adoption.