Engaging Multidisciplinary Health System Stakeholders to Create a Process for Implementing Machine-Learning Enabled Clinical Decision Support

项目总结/摘要 用于临床决策支持（CDS）的“黑匣子”机器学习（ML）模型的激增引发了 关于CDS的可解释性、可操作性和整体可用性的问题， 在实施计划中，涉及各种利益相关者（包括开发人员和用户）的过程。 我们的长期目标是正式确定一个程序，指导卫生系统进行规划、监测和评估 CDS实施。R21的总体目标是制定和评估一项可推广的战略 在CDS探索阶段将多学科利益相关者聚集在一起，以确定 促进者和障碍。在这样做时，我们将使用 动力学（PSD）建模作为一种多组件策略，用于评估和计划实施， 在实施的探索阶段，当决策发生时， ML使能的CDS可以随时间持续。因此，我们将侧重于上游实施 ML使能CDS的可接受性、适当性和可行性结果。该项目的基本原理是 一个让不同利益攸关方尽早参与实施规划的进程将明确承诺， 通过揭示可接受性、可行性和适当性来实现和采用的可能性。为 在本项目中，我们将重点关注一组特定的ML启用CDS：早期预警评分（EWS），用于 识别失代偿病人我们计划通过追求两个具体目标来实现我们的总体目标：1. 让参与EWS实施的多学科利益相关者（用户、开发人员、实施者， 所有者）从两个系统不同的采用环境中共同定义共同的障碍和促进者， 使用组模型构建CDS可接受性、适当性和可行性的实施结果 来自系统动力学领域的脚本和2.通过测量以下方面的变化来评估PSD过程 承诺采用CDS（使用可接受性，适当性和可行性的措施）， 反馈和估计干预努力。我们将通过一系列小组建模会议获得数据， 在不同环境中使用CDS的利益相关者，其中警报因目标用户、时间和频率而异 以及其他因素。我们将采用来自系统动力学建模领域的定义良好的脚本， 促进小组讨论，以发展关于ML启用CDS响应问题的共享理论 (Aim 1）。由于任何战略的实施都需要调整，我们将评估PSD过程（Aim 2)以改进和准备在其他地方使用。这一贡献是重要的，因为EWS被广泛使用 在学术和社区医院都有。这一贡献是创新性的使用群体建模 技术的问题ML启用CDS的实施，创建方法和实质性 调查结果。未来的R 01将前瞻性地评估在多个用例设置中使用此过程的好处 同时继续建立下游采用因素的动态系统模型。

PROJECT SUMMARY/ABSTRACT The proliferation of “black box” Machine Learning (ML) models for Clinical Decision Support (CDS) has raised concerns regarding CDS interpretability, actionability and overall usability, rendering a critical need for a clear process that engages various stakeholders including both developers and users in implementation planning. Our long-term goal is to formalize a process to guide health systems in planning, monitoring and evaluating CDS implementation. The overall objective for this R21 is to develop and evaluate a generalizable strategy to bring multidisciplinary stakeholders together during the CDS exploration phase to identify facilitators and barriers to implementation in their contexts. In doing so, we will use Participatory System Dynamics (PSD) modeling as a multi-component strategy to evaluate and plan implementation with stakeholders during the exploration phase of implementation, when decision-making occurs, in a way where ML-enabled CDS can be sustained over time. As such, we will focus on the upstream implementation outcomes of acceptability, appropriateness, and feasibility of ML-enabled CDS. The rationale for this project is that a process that engages diverse stakeholders in implementation planning early on will clarify commitment to implementation and potential for adoption by revealing acceptability, feasibility, and appropriateness. For this project we will focus on one particular set of ML-enabled CDS: Early Warning Scores (EWSs), used to identify decompensating patients. We plan to accomplish our overall objective by pursuing two specific aims: 1. Engage multidisciplinary stakeholders involved in EWS implementation (users, developers, implementers, owners) from two systematically varying adoption contexts to co-define common barriers and facilitators to key implementation outcomes of CDS acceptability, appropriateness, and feasibility using group model building scripts from the field of system dynamics and 2. Evaluate the PSD process by measuring change in commitment to adopt CDS (using measures of acceptability, appropriateness, and feasibility), eliciting feedback, and estimating intervention effort. We will obtain data via a series of group modeling sessions from stakeholders who have used CDS in different contexts, where alerts vary by target user, time, and frequency among other factors. We will employ well-defined scripts from the field of System Dynamics modeling to facilitate group discussion toward developing a shared theory about the problem of ML-enabled CDS response (Aim 1). Because implementation of any strategy requires adaptation, we will evaluate the PSD process (Aim 2) to refine and prepare for use elsewhere. This contribution is significant because EWSs are widely used across both academic and community hospitals. This contribution is innovative by using group modeling techniques for the problem of ML-enabled CDS implementation, creating both methodological and substantive findings. A future R01 will prospectively assess benefits of using this process in multiple use case settings while continuing to build out the dynamic systems model of factors for downstream adoption.