Collaborative Research: SCH: An AI Coach for Enhancing Teamwork in the Cardiac Operating Room

心脏手术通常需要解决一些最严重的心脏问题，导致每年超过90万例心脏手术。心脏手术室（OR）是一个复杂的环境，来自多个学科的医疗保健专业人员-包括外科医生，麻醉师，灌注师和护士-合作管理这种生命攸关的护理。为了成功地管理护理，手术团队的所有成员都应该步调一致地执行任务，并充分了解手术过程中遇到的动态情况。然而，在心脏手术室的复杂环境中，实现这种理想的团队合作是困难的，其中人的表现受到诸如高工作量、疲劳和手术期间的中断或中断等因素的不利影响。该项目通过设计人工智能（AI）支持的教练系统（AI Coach）来监测，评估和增强心脏手术室的手术团队合作，从而解决了减轻这些可预防的人为错误并提高患者安全性的迫切需求。 人工智能教练的核心功能将是一套新颖的机器学习和可解释的人工智能算法，以计算方式生成可解释的反馈和干预措施，以增强基于多模态传感器数据的手术团队合作。该项目将培养学生在智能健康的多学科研究领域。该项目将通过将研究成果纳入计划中的人类-人工智能合作博物馆展览，增加公众对人工智能的参与。该项目的总体目标是设计由多模态传感硬件、数据驱动算法和用户界面组成的AI教练系统，以增强心脏手术室的手术团队合作。AI Coach将通过两种并行策略实现其目标：（i）解决建模手术团队合作的问题;（ii）通过计算生成反馈以改善这种团队合作。该项目团队将首先开发一个新的团队马尔可夫模型（TMkM），反映了手术团队的心理模型。然后，该系统的计算核心将通过开发（a）基于新型多智能体模仿学习方法的机器学习算法来实现，以达到明确依赖于潜在性能塑造因素（如心理模型）的团队合作预测模型，以及（ii）可解释的人工智能技术，以计算方式生成可解释的反馈和干预措施，以增强团队合作。由于收集手术团队的大型数据集的挑战，算法开发将强调样本和标签效率的技术。项目小组将采用迭代的、以用户为中心的设计方法，对综合系统进行原型设计和可用性测试。该解决方案将使用多模式专家注释的手术团队数据进行开发和评估，并在最先进的OR模拟设施中进行原型制作。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Cardiac surgery is often needed to address some of the most serious heart problems, resulting in administration of more than 900,000 cardiac procedures each year. The cardiac Operating Room (OR) is a complex environment where healthcare professionals from multiple disciplines -- including surgeons, anesthesiologists, perfusionists, and nurses -- collaborate to administer this life-critical care. To successfully administer care, all members of the surgical team are expected to perform their tasks in lockstep and with full awareness of dynamic situations encountered during surgery. However, achieving such ideal teamwork is difficult in the complex environment of cardiac OR, where human performance is adversely affected by factors such as high workload, fatigue, and interruptions or disruptions during surgery. This project addresses an urgent need for mitigating these preventable human errors and improving patient safety through the design of an Artificial Intelligence (AI)-enabled coaching system (AI Coach) for monitoring, assessing, and enhancing surgical teamwork in the cardiac OR. Central to the functioning of the AI Coach will be a set of novel machine learning and explainable artificial intelligence algorithms to computationally generate interpretable feedback and interventions for enhancing surgical teamwork based on multimodal sensor data. The project will train students in the multi-disciplinary research area of Smart Health. The project will increase public engagement with AI, by incorporating the research results into a planned museum exhibit on human-AI collaboration. The project’s overarching goal is to design the AI Coach system comprised of multimodal sensing hardware, data-driven algorithms, and a user interface to enhance surgical teamwork in the cardiac OR. AI Coach will achieve its objectives by pursuing two parallel strategies: (i) addressing the problem of modeling surgical teamwork; (ii) computationally generating feedback to improve this teamwork. The project team will first develop a novel Team Markov Model (TMkM) that reflects the surgical team’s mental model. Then, the computational core of the system will be realized through the development of (a) machine learning algorithms based on novel multi-agent imitation learning methods to arrive at predictive models of teamwork that explicitly depend on latent performance-shaping factors, such as mental models, and (ii) explainable AI techniques to computationally generate interpretable feedback and interventions for enhancing teamwork. Due to the challenge of collecting large data sets of surgical teamwork, the algorithm development will emphasize sample- and label-efficient techniques. The project team will prototype and test usability of the integrated system by employing iterative, user-centered design approaches. The solutions will be developed and evaluated using multi-modal expert-annotated data of surgical teamwork and prototyped in a state-of-the-art OR simulation facility.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.