A robust, reliable and multimodal AI system for pain quantification

该项目涉及开发一个强大、可靠和多模式的人工智能系统，用于疼痛量化，该系统基于上一个项目中获得的X-ITE数据库。主要重点是提高治疗质量，并在手术干预后的恢复和监测阶段加强痴呆症患者的健康和生活质量。它还旨在通过自动化实时疼痛监测，支持和减轻医务人员的疼痛评估，以及实现更精确、个性化和具体情况的镇痛，从而更好地治疗疼痛及其原因。特别是在痴呆症患者中，疼痛识别相当困难，因为受影响的人通常会忘记他们的痛苦或失去口头表达的能力。由于这些认知限制，外部评估工具应该用于这些患者的疼痛识别，因为自我报告作为金标准在这里不能提供可靠的信息。关于自动疼痛识别，在数据（疼痛强度的客观记录、外观变化、面部表情、头部姿势、照明或部分遮挡）和算法（面部表情的解释、样本的有限可用性、模型的选择和类别的不均匀分布）方面出现了各种挑战。这些挑战在以下子目标中得到解决：使用深度神经网络和迁移学习与大型现有的野外数据库，以增加对外观，照明等差异的鲁棒性。评估以前未使用的身体姿势、热图像和头部侧面视图的模式，以及融合不同模式的复杂方法，2。3 .结合深度学习和时间评估，例如，通过使用时间卷积网络（TCN）或变压器；当传感仅限于模态子集（例如，仅皮肤电导或仅非接触传感）时，研究可实现的测量质量。使用回归将实现疼痛强度的连续预测。通过将深度神经网络、变形器和长短期记忆（LSTM）与迁移学习相结合，以及通过面部表情来评估疼痛的附加标签，有望在疼痛识别的可靠性、准确性和鲁棒性方面得到显著提高，这将成为将该系统应用于临床实践的基础。

The project deals with the development of a robust, reliable and multimodal AI system for pain quantification based on the X-ITE database acquired in the previous project. The main focus is to improve the quality of treatment and the associated strengthening of the health and life quality of patients with dementia during the recovery and monitoring phase after surgical interventions. It also aims to enable better treatment of pain and its causes by supporting and relieving medical staff in pain assessment through automated real-time pain monitoring and by enabling more precise, individual and situation-specific analgesia. Especially in dementia patients, pain recognition is considerably more difficult, as the affected persons often forget their pain suffering or have lost the ability to verbally express it. Due to these cognitive limitations, external assessment instruments should be used for pain recognition for these patients, since self-report as the gold standard does not provide reliable information here. Regarding automatic pain recognition, various challenges arise in terms of data (objective recording of pain intensity, variations in appearance, facial expression, head pose, illumination, or partial occlusions) and algorithms (interpretation of facial expressions, limited availability of samples, choice of model, and unequal distribution of classes). These challenges are addressed in the following sub-objectives: 1. The use of deep neural networks and transfer learning with large existing in-the-wild databases to increase robustness to variances in appearance, illumination, etc., 2. evaluating previously unused modalities of body pose, thermal images, and side views of the head, as well as complex methods for fusing different modalities, 3. combining deep learning and temporal evaluation, for example, by using temporal convolutional networks (TCN) or transformers, and 4. investigating the achievable quality of measurement when sensing is limited to subsets of modalities (e.g., skin conductance only or noncontact sensing only). Continuous prediction of pain intensities will be enabled using regression. By using deep neural networks, transformers and Long Short-Term Memory (LSTM) in combination with transfer learning, as well as additional labels evaluating pain expressed by facial expressions, a significant improvement of pain recognition in terms of reliability, accuracy and robustness is expected, which should form the basis for transferring the system into clinical practice.