Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
复制标题
DOI：
10.1145/3292039
复制
发表时间：
2016-06
期刊：
影响因子：
--
通讯作者：
Justus Thies;M. Zollhöfer;M. Stamminger;C. Theobalt;M. Nießner
中科院分区：
文献类型：
--
作者：
Justus Thies;M. Zollhöfer;M. Stamminger;C. Theobalt;M. Nießner
提出了一种实时再现单目目标视频序列(如YouTube视频)的人脸再现方法。源序列也是用商用网络摄像头实时捕捉的单目视频流。我们的目标是通过源演员使目标视频的面部表情动画化，并以照片逼真的方式重新渲染操纵后的输出视频。为此，我们首先解决了基于非刚性模型捆绑的单目视频人脸身份恢复的欠约束问题。在运行时，我们使用密集光度一致性度量来跟踪源和目标视频的面部表情。然后，通过源和目标之间快速高效的变形传输来实现重演。从靶序列中检索与重新定位的表达最匹配的嘴部内部，并对其进行扭曲以产生精确的匹配。最后，我们令人信服地在相应的视频流上重新绘制合成的目标人脸，使其与真实世界的光照无缝融合。我们在实时设置中演示了我们的方法，其中YouTube视频被实时重演。
We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.