Collaborative Research: SCH: An AI Coach for Enhancing Teamwork in the Cardiac Operating Room

心脏手术通常需要解决一些最严重的心脏问题，每年有超过90万例心脏手术。心脏手术室（OR）是一个复杂的环境，来自多个学科的医疗保健专业人员——包括外科医生、麻醉师、灌注师和护士——合作管理这种生命攸关的护理。为了成功地管理护理，外科团队的所有成员都应该步调一致地完成任务，并充分意识到手术过程中遇到的动态情况。然而，在心脏手术室的复杂环境中，实现这种理想的团队合作是困难的，在这种环境中，人的表现会受到高工作量、疲劳和手术期间中断或中断等因素的不利影响。该项目通过设计一个人工智能（AI）指导系统（AI Coach）来监测、评估和加强心脏手术室的手术团队合作，解决了减轻这些可预防的人为错误和提高患者安全性的迫切需求。AI教练的核心功能将是一套新颖的机器学习和可解释的人工智能算法，以计算生成可解释的反馈和干预，以增强基于多模态传感器数据的手术团队合作。该项目将在智能健康的多学科研究领域培养学生。该项目将通过将研究成果纳入计划中的人类与人工智能合作博物馆展览，增加公众对人工智能的参与。该项目的总体目标是设计由多模态传感硬件、数据驱动算法和用户界面组成的人工智能教练系统，以加强心脏手术室的手术团队合作。AI Coach将通过采取两种并行策略来实现其目标：(i)解决手术团队合作的建模问题；（ii）通过计算生成反馈以改进这种团队合作。项目团队将首先开发一个新的团队马尔可夫模型（TMkM），它反映了外科团队的心理模型。然后，系统的计算核心将通过开发(a)基于新型多智能体模仿学习方法的机器学习算法来实现，以获得明确依赖于潜在绩效塑造因素（如心理模型）的团队合作预测模型，以及（ii）可解释的人工智能技术来计算生成可解释的反馈和干预措施，以增强团队合作。由于收集大量手术团队数据集的挑战，算法开发将强调样本和标签效率技术。项目团队将通过采用迭代的、以用户为中心的设计方法，对集成系统的可用性进行原型和测试。解决方案的开发和评估将使用手术团队的多模式专家注释数据，并在最先进的手术室模拟设施中进行原型设计。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Cardiac surgery is often needed to address some of the most serious heart problems, resulting in administration of more than 900,000 cardiac procedures each year. The cardiac Operating Room (OR) is a complex environment where healthcare professionals from multiple disciplines -- including surgeons, anesthesiologists, perfusionists, and nurses -- collaborate to administer this life-critical care. To successfully administer care, all members of the surgical team are expected to perform their tasks in lockstep and with full awareness of dynamic situations encountered during surgery. However, achieving such ideal teamwork is difficult in the complex environment of cardiac OR, where human performance is adversely affected by factors such as high workload, fatigue, and interruptions or disruptions during surgery. This project addresses an urgent need for mitigating these preventable human errors and improving patient safety through the design of an Artificial Intelligence (AI)-enabled coaching system (AI Coach) for monitoring, assessing, and enhancing surgical teamwork in the cardiac OR. Central to the functioning of the AI Coach will be a set of novel machine learning and explainable artificial intelligence algorithms to computationally generate interpretable feedback and interventions for enhancing surgical teamwork based on multimodal sensor data. The project will train students in the multi-disciplinary research area of Smart Health. The project will increase public engagement with AI, by incorporating the research results into a planned museum exhibit on human-AI collaboration. The project’s overarching goal is to design the AI Coach system comprised of multimodal sensing hardware, data-driven algorithms, and a user interface to enhance surgical teamwork in the cardiac OR. AI Coach will achieve its objectives by pursuing two parallel strategies: (i) addressing the problem of modeling surgical teamwork; (ii) computationally generating feedback to improve this teamwork. The project team will first develop a novel Team Markov Model (TMkM) that reflects the surgical team’s mental model. Then, the computational core of the system will be realized through the development of (a) machine learning algorithms based on novel multi-agent imitation learning methods to arrive at predictive models of teamwork that explicitly depend on latent performance-shaping factors, such as mental models, and (ii) explainable AI techniques to computationally generate interpretable feedback and interventions for enhancing teamwork. Due to the challenge of collecting large data sets of surgical teamwork, the algorithm development will emphasize sample- and label-efficient techniques. The project team will prototype and test usability of the integrated system by employing iterative, user-centered design approaches. The solutions will be developed and evaluated using multi-modal expert-annotated data of surgical teamwork and prototyped in a state-of-the-art OR simulation facility.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.