Bridging the skin: Facial surface and mimic muscles as a unit: Fully automated classification of motoric function and emotional expression in patients with facial palsy (BRIDGING THE GAP: MIMICS AND MUSCLES)

面瘫是最常见的脑神经麻痹或麻痹。面神经麻痹在大多数情况下会导致单侧的模仿肌运动功能障碍。例如，不完全的眼睛闭合或眨眼导致干眼症，口周肌肉功能障碍导致食物摄入受损和说话受损。情绪表达的能力受到干扰对患者来说是必不可少的，例如，因为通常的微笑不再可能。最佳治疗的目标是运动功能（临床常规的标准目标）和情感表达（迄今经常被忽视）的良好康复。临床常规中对疾病严重程度进行分类以及监测治疗变化的标准主要是主观的，因此是不可靠和不精确的评估工具。该建议的目的是开发一种客观的测量系统，使用面部的自动图像分析来描述运动障碍，同时描述情感表达的缺陷。到目前为止，大多数用于分析面瘫的自动化方法都集中在2D数据集的静态评估或3D点云序列表面上的不对称性评估上。迄今为止，只有少数概念采用了公认的客观量化措施。根本没有分析对情感表达的影响。在本提案中，健康受试者和患者的模拟运动和情感表达的标准化3-D视频记录将被用作训练数据集。肌肉模型将通过融合模拟肌肉的肌电图记录的测量结果，同时执行标准化运动以及面部的同步3D记录。此外，来自面部肌肉的超声记录的数据将作为预先存在的知识被包括到模型中。情绪表现力的变化及其对生活质量的影响将通过标准问卷进行评估，并最终与客观和自动分类的运动数据合并为一个整体指标。使用无监督学习的算法组，即所谓的生成对抗网络，以及训练数据的合成积累，这将是第一次使用深度学习方法来改善面瘫的治疗。该项目获得的发现不仅与面瘫有关，而且一般与任何具有模仿肌肉功能或情感表达障碍的疾病有关。本文获得的面瘫患者的结果可以作为一个模型。

Facial palsy is the most common paresis or paralysis of a cranial nerve. A facial palsy results in most cases in a unilateral motoric dysfunction of the mimic muscles. For instance, incomplete eye closure or blinking is leading to a dry eye and dysfunction of the perioral muscles is leading to impaired intake of food and impaired speaking. The disturbed capability for emotional expressions is essential for the patients, for instance, because usual smiling is not possible anymore. Goal of optimal treatment is good rehabilitation of motoric function (standard goal in clinical routine) and of emotional expressiveness (hitherto often neglected). Standard in clinical routine to classify the severity of the disease as well as to monitor changes under therapy are mainly subjective and therefore unreliable and imprecise assessment tools. Aim of this proposal is to develop an objective measurement system using automated image analysis of the face for the description of the motoric disturbances and at the same time of the deficits in emotional expression. So far, most automated approaches for analysis of facial palsy were focused on static evaluations of 2D-datasets or evaluation of asymmetry on the surface of sequences of 3D-point clouds. To date, only a few concepts used accepted objective quantitative measures. The impact on the emotional expressiveness was not analyzed at all. In the present proposal standardized 3-D video recordings of mimic movements and emotional expressions of healthy subjects and patients will be used as training datasets. A muscle model will be included by fusion of measurements of electromyography recording of mimic muscles while performing standardized movements together with synchronous 3D-recordings of the face. Furthermore, data from sonography recordings of facial muscles will be included into the model as preexisting knowledge. The changes of emotional expressiveness and consequences on quality of life will be assessed by standard questionnaires and finally merged with the objective and automatically classified movement data to an overall index. Using groups of algorithms for unsupervised learning, so called generative adversarial networks, and synthetic accumulation of training data, it will be the first time to use deep learning approaches to improve the treatment of facial palsy. The findings gained by this project are not only relevant for facial palsy, but in general for any disease with disturbance of mimic muscle function or emotional expression. The results obtained herein for patients with facial palsy can serve as a model.