Real-Time 4D Facial Sensing and Modelling

实时 4D 面部传感和建模

• 批准号: EP/N025849/1
• 负责人: Hui Yu
• 金额: $ 12.82万
• 依托单位: University of Portsmouth
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2016
• 资助国家: 英国
• 起止时间: 2016 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FN025849%2F1
• 关键词: Real; Time; 4D; Facial; Sensing

Chronic Facial Palsy is caused by conditions including Bell's palsy and Stroke, afflicting, at a conservative estimate, 51,640 new people per year in the UK. Estimates based on UK incidence data for Bell's palsy patients and those left with residual weakness requiring specialist management (approximately 30%) suggests that to treat all new patients could cost between £2,543,000 (therapy alone) to £7,948,000 (therapy plus botulinum toxin injections) per annum. Approximately 30% of patients affected by facial palsy suffer ongoing chronic disfigurement, anxiety and/or depression. It has been shown that service provision for these patients is limited. Facial palsy management is expensive. With the NHS requiring to save billions annually (NHS, 2011), it is imperative to rethink conventional treatment pathways. The NHS National Clinical Guidelines for Stroke physiotherapy recommends 45 minutes daily facial therapy for patients with facial paralysis. To meet the guidelines, each patient would need daily face-to-face therapy for a period of at least 12 weeks (broadly defined as the acute phase), costing the NHS £2,400 per patient. With 26,000 new cases annually this would represent a prohibitive cost of £62,400,000 for these new patients per annum on top of the cost of treating existing patients. However, costs could be reduced by developing a home-based rehabilitative technology, allowing greater numbers of patients to receive gold-standard treatment. The proposed technologies will provide patients with real-time feedback when undergoing therapy at home and thus can significantly reduce the time of visiting therapists for face-to-face feedback.This project will investigate 4D (dynamic three-dimension) techniques and a computational model integrating Mirror Visual Feedback MVF theory for home-based therapy using a depth sensor. This framework envisions developing easy to use facial palsy therapy technologies, which can provide real-time feedback assessing treatment responses of patients integrating MVF theory. Research studies using MVF therapy to treat phantom limb pain and complex regional pain syndrome have produced promising results. Facial palsy patients can recover more quickly if they exercise their facial muscles. However, it can be painful for patients to face mirrors due to the anxiety of looking at their asymmetric and deformed facial features. Therefore, we will apply MVF to adaptively mirror the healthy side of the face and facial movement over the unhealthy side and thus allow patients to observe healthy whole faces when exercising their facial muscles. It has been reported that biofeedback therapy based on MVF has been effective for Bell's facial palsy and hemi-facial pain of trigeminal neuralgia. However, these studies either use a physical mirror box or simple image-based mapping, which provide little feedback or inaccurate facial movement information. Patients have limited awareness of the abnormal movements their faces display so without feedback about these movements their facial functions may worsen, developing permanently abnormal movements. Therefore, there is a strong need for novel computational 4D sensing and modelling methods to develop therapy which can capture and map accurate facial muscle movements according to MVF. The ultimate goal is to programme this method into a software package for home-based therapy. Currently, there are no 3D or 4D products or technologies based on MVF available for therapy. Thus, the proposed methods for building computational 4D sensing and modelling models integrating MVF will be hugely beneficial to patients and the NHS by providing appropriate feedback in assessing treatment responses of patients and ultimately improve the ability to scale home-based therapy. It can also be adapted to tele-rehabilitation for practical applications so clinical consultants and therapists can remotely observe patients' home-based therapy.

慢性面瘫是由贝尔氏麻痹和中风等疾病引起的，保守估计，英国每年有51，640名新患者受到影响。根据英国贝尔氏麻痹患者和需要专科治疗的残留无力患者（约30%）的发病率数据估计，治疗所有新患者每年的费用可能在2，543，000英镑（仅治疗）至7，948，000英镑（治疗加肉毒杆菌毒素注射）之间。大约30%的面瘫患者患有持续的慢性毁容，焦虑和/或抑郁。事实证明，为这些病人提供的服务是有限的。面瘫的治疗是昂贵的。随着NHS每年需要节省数十亿美元（NHS，2011），必须重新考虑传统的治疗途径。NHS国家中风理疗临床指南建议面瘫患者每天进行45分钟的面部治疗。为了满足指南，每位患者将需要每天面对面治疗至少12周（广义上定义为急性期），每位患者的NHS费用为2，400英镑。由于每年有26，000例新病例，这将意味着除了治疗现有患者的费用之外，每年这些新患者的费用高达62，400，000英镑。然而，可以通过开发基于家庭的康复技术来降低成本，使更多的患者能够接受金标准治疗。该技术将为患者在家中接受治疗时提供实时反馈，从而大大减少访问治疗师进行面对面反馈的时间。本项目将研究4D（动态三维）技术和集成镜像视觉反馈MVF理论的计算模型，用于使用深度传感器的家庭治疗。该框架设想开发易于使用的面瘫治疗技术，该技术可以提供实时反馈，评估患者的治疗反应，并结合MVF理论。使用MVF疗法治疗幻肢痛和复杂区域疼痛综合征的研究已经产生了有希望的结果。面神经麻痹患者如果锻炼面部肌肉，可以恢复得更快。然而，患者面对镜子可能会感到痛苦，因为他们对自己不对称和变形的面部特征感到焦虑。因此，我们将应用MVF来自适应地反映面部的健康侧和面部运动，从而使患者在锻炼面部肌肉时能够观察到健康的整个面部。已有文献报道，基于MVF的生物反馈治疗对Bell面瘫和三叉神经痛的半面痛有效。然而，这些研究要么使用物理镜盒，要么使用简单的基于图像的映射，这提供了很少的反馈或不准确的面部运动信息。患者对他们面部显示的异常运动的意识有限，因此如果没有关于这些运动的反馈，他们的面部功能可能会恶化，发展出永久性的异常运动。因此，迫切需要新的计算4D感测和建模方法来开发可以根据MVF捕获和映射准确的面部肌肉运动的治疗。最终的目标是将这种方法编程到一个软件包中，用于家庭治疗。目前，没有基于MVF的3D或4D产品或技术可用于治疗。因此，所提出的用于构建集成MVF的计算4D感测和建模模型的方法将通过在评估患者的治疗反应时提供适当的反馈并最终提高扩展基于家庭的治疗的能力而对患者和NHS非常有益。它还可以适用于实际应用的远程康复，因此临床顾问和治疗师可以远程观察患者的家庭治疗。

项目成果

Saliency detection by conditional generative adversarial network

• DOI: 10.1117/12.2306421
• 发表时间: 2018-04
• 作者: Xiaoxu Cai;Hui Yu

Real-Time Facial Affective Computing on Mobile Devices

• DOI: 10.3390/s20030870
• 发表时间: 2020-02-01
• 期刊: SENSORS
• 影响因子: 3.9
• 作者: Guo, Yuanyuan;Xia, Yifan;Chen, Rung-Ching
• 通讯作者: Chen, Rung-Ching

Real-time 3D Facial Tracking via Cascaded Compositional Learning

通过级联组合学习进行实时 3D 面部跟踪

• DOI: 10.48550/arxiv.2009.00935
• 发表时间: 2020
• 作者: Lou J

Dynamic 3D Surface Reconstruction Using a Hand-Held Camera

使用手持式相机进行动态 3D 表面重建

• 发表时间: 2018
• 作者: Fan H

Perceptual loss guided Generative adversarial network for saliency detection

• DOI: 10.1016/j.ins.2023.119625
• 发表时间: 2023-09
• 期刊: Inf. Sci.
• 作者: Xiaoxu Cai;Gaige Wang;Jianwen Lou;Muwei Jian;Junyu Dong;Rung-Ching Chen;Brett Stevens;Hui Yu