Multimodal AI-based pain measurement in intermediate care patients in the postoperative period

该项目的目的是使用多模态传感器和人工智能方法自动连续测量术后住院患者的疼痛强度，并将进一步开发和验证用于临床。这就产生了一种用于实时疼痛监测的新型系统，它使医务人员从日常工作中解脱出来，并在必要时提醒专家。该项目分为以下阶段：“测试”，第一阶段+第二阶段：“翻译和数据收集”和第三阶段：“验证”。测试：多模式测量基础设施将在实验室环境中开发。为此，必要的红外摄像机技术（用于捕捉面部表情和手势），将安装生物信号放大器和数据同步和记录技术，并将建立一个与乌尔姆大学医院重症监护室的房间相当的实验室，即具有相同的重症监护床，床相对于墙壁的相同布置，天花板灯和/或“假人”形式的医疗技术等。第一阶段：将测量基础设施集成到乌尔姆大学医院的跨学科手术重症监护医学的病房中。随后，对84名全方位患者进行多模式数据收集-“黄金标准”（人工智能地面实况）这里是主观疼痛感知以及外部观察，在每种情况下，在计划的操作之前进行基线测量，以及在操作之后48小时内进行连续测量-并记录在一段时间内的疼痛，包括药物疼痛治疗的剂量。与此同时，在实验室中开发的多模式疼痛检测算法正在适应新的要求。这包括应用深度迁移学习来调整计算机视觉程序，从彩色到NIR视频，调整实时数据分析程序，以及迁移学习来调整疼痛检测模型，从我们的初步工作到临床条件，要求和疼痛模式。深度学习能够学习针对应用优化的特征提取，并且在与迁移学习结合时具有巨大的潜力。第二阶段：在另一个非全方位患者样本（N=42）中，也将收集和分析多模态数据，此处的“金标准”是“仅”第三方观察。第三阶段：将开发演示器，并使用全方位和非全方位定向的另一组患者（N=10）进行验证。一个算法将检测真实的时间疼痛。还将进行医务人员对开发的演示器的验收调查。研究结果将发表在期刊上并在会议上发表。

The aim of the project is the automated and continuous measurement of pain intensity in patients in postoperative inpatient care using multimodal sensors and artificial intelligence methods, which will be further developed and validated for clinical use. This results in a novel system for real-time pain monitoring, which relieves medical staff from routine tasks and can alert experts if necessary. The project is divided into the following phases: "Testing", Phase I+II: "Translation & Data Collection" and Phase III: "Validation". Testing: The multimodal measurement infrastructure will be developed in a laboratory environment. For this purpose, the necessary infrared camera technology (for capturing facial expressions and gestures), biosignal amplifiers and data synchronization and recording technology will be installed and a laboratory room comparable to a room in the intensive care unit of the University Hospital Ulm will be set up, i.e. with an identical intensive care bed, the same arrangement of the bed with respect to the wall, ceiling lights and/or medical technology in the form of "dummies", etc. Phase I: The measurement infrastructure is integrated into a patient room of the Interdisciplinary Operative Intensive Care Medicine of the University Hospital Ulm. Subsequently, the multimodal data collection with 84 all-round oriented patients takes place - the "gold standard" (AI ground truth) here is the subjective pain perception as well as external observation, whereby in each case a baseline measurement is carried out before the planned operation as well as a continuous measurement over 48 hours after the operation - with recording of the pain in the course of time including the dosage of the medicinal pain therapy. In parallel, algorithms for the multimodal detection of pain, which have been developed in the laboratory, are being adapted to the new requirements. This includes the application of Deep Transfer Learning to adapt computer vision procedures from color to NIR video, adaptation of procedures for real-time data analysis, and transfer learning to adapt pain detection models from our preliminary work to clinical conditions, requirements, and pain modalities. Deep learning enables learning of feature extraction optimized for the application and has great potential when combined with transfer learning. Phase II: In a further sample of not omnidirectional patients (N=42) multimodal data will be collected and analyzed as well, the "gold standard" here is "only" third-party observation. Phase III: A demonstrator will be developed and validated with a further group of patients (N=10) who are both omnidirectionally and non-omnidirectionally oriented. An algorithm will detect pain in real time. An acceptance survey of the developed demonstrator by medical staff will also be conducted. Results will be published in journals and presented at conferences.