BioFace-3D: continuous 3d facial reconstruction through lightweight single-ear biosensors
BioFace-3D: continuous 3d facial reconstruction through lightweight single-ear biosensors
复制标题
BioFace-3D：通过轻型单耳生物传感器进行连续 3D 面部重建
DOI：
10.1145/3447993.3483252
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Nguyen, Phuc
中科院分区：
文献类型：
--
作者：
Wu, Yi;Kakaraparthi, Vimal;Li, Zhuohang;Pham, Tien;Liu, Jian;Nguyen, Phuc
在过去的十年中，面部标志跟踪和3D重建由于其诸如人机交互、面部表情分析和情感识别等的众多应用而获得了相当大的关注。传统方法要求用户被限制在特定位置并且在受限的记录条件下面对相机（例如，没有遮挡并且在良好的照明条件下）。这种高度受限的设置使它们无法部署在涉及人体运动的许多应用场景中。在本文中，我们提出了第一个单耳机轻量级生物传感系统，BioFace-3D，可以不引人注目，连续，可靠地感知整个面部运动，跟踪2D面部标志，并进一步渲染3D面部动画。我们的单耳机生物传感系统利用跨模态迁移学习模型将高级视觉面部标志检测模型中包含的知识转移到低级生物信号域。经过训练后，我们的BioFace-3D可以直接从生物信号中进行连续的3D面部重建，而无需任何视觉输入。在不需要将摄像头置于用户面前的情况下，这种从视觉传感到生物传感的范式转变将为许多新兴的移动的和物联网应用带来新的机会。涉及16名参与者在各种设置下的广泛实验表明，BioFace-3D可以准确跟踪53个主要面部标志，平均误差仅为1.85 mm，标准化平均误差为3.38%，与大多数最先进的基于摄像头的解决方案相当。绘制的三维人脸动画与真实的人脸运动一致，验证了系统的连续三维人脸重建能力。
Over the last decade, facial landmark tracking and 3D reconstruction have gained considerable attention due to their numerous applications such as human-computer interactions, facial expression analysis, and emotion recognition, etc. Traditional approaches require users to be confined to a particular location and face a camera under constrained recording conditions (e.g., without occlusions and under good lighting conditions). This highly restricted setting prevents them from being deployed in many application scenarios involving human motions. In this paper, we propose the first single-earpiece lightweight biosensing system,BioFace-3D, that can unobtrusively, continuously, and reliably sense the entire facial movements, track 2D facial landmarks, and further render 3D facial animations. Our single-earpiece biosensing system takes advantage of the cross-modal transfer learning model to transfer the knowledge embodied in ahigh-gradevisual facial landmark detection model to thelow-gradebiosignal domain. After training, ourBioFace-3Dcan directly perform continuous 3D facial reconstruction from the biosignals, without any visual input. Without requiring a camera positioned in front of the user, this paradigm shift from visual sensing to biosensing would introduce new opportunities in many emerging mobile and IoT applications. Extensive experiments involving 16 participants under various settings demonstrate thatBioFace-3Dcan accurately track 53 major facial landmarks with only 1.85 mm average error and 3.38% normalized mean error, which is comparable with most state-of-the-art camera-based solutions. The rendered 3D facial animations, which are in consistency with the real human facial movements, also validate the system's capability in continuous 3D facial reconstruction.
登录
查看更多内容
影响因子：
4.6
作者：
WEBSTER, JG
通讯作者：
WEBSTER, JG
使用双通道生物传感器配置和模糊分类器的面部手势识别：试点研究
DOI：
--
发表时间：
2011
期刊：
International Conference on Electrical, Control and Computer Engineering
影响因子：
--
作者：
M. Hamedi;I. Mohammad Rezazadeh;M. Firoozabadi
通讯作者：
M. Firoozabadi
基于磁传感器的可穿戴式无声语音接口，具有运动伪影消除功能
DOI：
--
发表时间：
2018
期刊：
International Conference on Biomedical Electronics and Devices
影响因子：
--
作者：
L. A. Cheah;J. M. Gilbert;José A. González;P. Green;S. Ell;Roger K. Moore;E. Holdsworth
通讯作者：
E. Holdsworth
一种基于多通道面部生物电信号识别的新型人机界面
影响因子：
--
作者：
Iman Mohammad;Rezazadeh bullet;S. Mohammad;F. bullet;Huosheng Hu;bullet S Mohammad;Reza Golpayegani
通讯作者：
Reza Golpayegani