Augmented Blendshapes for Real-Time Simultaneous 3D Head Modeling and Facial Motion Capture
Augmented Blendshapes for Real-Time Simultaneous 3D Head Modeling and Facial Motion Capture
复制标题
DOI：
10.1109/cvpr.2016.359
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
D. Thomas;R. Taniguchi
中科院分区：
文献类型：
--
作者：
D. Thomas;R. Taniguchi
我们提出了一种使用消费级RGB-D相机构建实时动画3D头部模型的方法。我们的框架是第一个同时提供全面的面部运动跟踪和用户头部的详细3D模型的框架。任何人的头部都可以立即重建，并且无需任何训练或预先扫描即可捕获他的面部动作。在第一帧中，用户开始以中性的表情面对摄像机，但可以自由移动、说话和改变面部表情。使用混合形状表示跟踪面部运动，而使用映射在模板网格上的Bump图像捕获精细的几何细节。我们提出了一种有效的算法来生长和完善三维模型的头部在飞行和实时。我们展示了鲁棒和高保真的同步面部运动跟踪和3D头部建模结果在广泛的主题与各种头部姿势和面部表情。我们提出的方法为动画制作和3D视频通信提供了有趣的可能性。
We propose a method to build in real-time animated 3D head models using a consumer-grade RGB-D camera. Our framework is the first one to provide simultaneously comprehensive facial motion tracking and a detailed 3D model of the user's head. Anyone's head can be instantly reconstructed and his facial motion captured without requiring any training or pre-scanning. The user starts facing the camera with a neutral expression in the first frame, but is free to move, talk and change his face expression as he wills otherwise. The facial motion is tracked using a blendshape representation while the fine geometric details are captured using a Bump image mapped over the template mesh. We propose an efficient algorithm to grow and refine the 3D model of the head on-the-fly and in real-time. We demonstrate robust and high-fidelity simultaneous facial motion tracking and 3D head modeling results on a wide range of subjects with various head poses and facial expressions. Our proposed method offers interesting possibilities for animation production and 3D video telecommunications.