Data-driven extraction and composition of secondary dynamics in facial performance capture
Data-driven extraction and composition of secondary dynamics in facial performance capture
复制标题
面部表情捕捉中二次动力学的数据驱动提取和合成
DOI：
10.1145/3386569.3392463
复制
发表时间：
2020
影响因子：
6.2
通讯作者：
Bradley, Derek
中科院分区：
文献类型：
--
作者：
Zoss, Gaspard;Sifakis, Eftychios;Gross, Markus;Beeler, Thabo;Bradley, Derek
表现性主体的表现捕捉，特别是以高空间分辨率获得的面部表现，将不可避免地包含由于惯性效应和由于弹道运动的动态过冲而引起的运动的一部分。这在大多数自然的捕捉环境中是真实的，其中演员能够在他们的表演期间自由移动，而不是被束缚在固定位置。通常，这些次级动态效果是不需要的，因为捕获的面部表演通常被重定向到不同的头部运动，并且有时被重定向到完全不同的角色，并且在这两种情况下，捕获的动态效果应该被移除并且新的次级效果应该被添加。本文提出了这样的假设：对于人脸等高度约束的弹性介质，这些二次惯性效应主要是由于底层骨骼结构（颅骨和下颌骨）的运动造成的。我们的工作旨在计算和描述捕获的动态面部表现之间的差异，以及如果惯性效应不存在，则相同运动的推测性准静态变体。这用于减去捕获期间由无意运动引起的寄生次级动态，或者在准静态表演之上合成这样的效果，以模拟演员的身体和头骨的新动态运动，无论是艺术家规定的还是经由运动捕获获得的。我们提出了一种数据驱动的技术，包括互补的删除和合成网络的二次动态面部表演捕捉。我们展示了这样一个系统如何可以有效地训练从不同的表达式下获得的动态变形的集合，演员诱导刚性头部运动从步行和跑步，以及强制振荡的身体运动在受控设置由外部致动器。
Performance capture of expressive subjects, particularly facial performances acquired with high spatial resolution, will inevitably incorporate some fraction of motion that is due to inertial effects and dynamic overshoot due to ballistic motion. This is true in most natural capture environments where the actor is able to move freely during their performance, rather than being tethered to a fixed position. Normally these secondary dynamic effects are unwanted, as the captured facial performance is often retargeted to different head motion, and sometimes to completely different characters, and in both cases the captured dynamic effects should be removed and new secondary effects should be added. This paper advances the hypothesis that for a highly constrained elastic medium such as the human face, these secondary inertial effects are predominantly due to the motion of the underlying bony structures (cranium and mandible). Our work aims to compute and characterize the difference between the captured dynamic facial performance, and a speculative quasistatic variant of the same motion should the inertial effects have been absent. This is used to either subtract parasitic secondary dynamics that resulted from unintentional motion during capture, or compose such effects on top of a quasistatic performance to simulate a new dynamic motion of the actor's body and skull, either artist-prescribed or acquired via motion capture. We propose a data-driven technique that comprises complementary removal and synthesis networks for secondary dynamics in facial performance capture. We show how such a system can be effectively trained from a collection of acquired dynamic deformations under varying expressions where the actor induces rigid head motion from walking and running, as well as forced oscillatory body motion in a controlled setting by external actuators.
DOI：
10.1145/1073368.1073412
发表时间：
2005-07
期刊：
--
影响因子：
--
作者：
Steve Capell;M. Burkhart;B. Curless;T. Duchamp;Zoran Popovic
通讯作者：
Steve Capell;M. Burkhart;B. Curless;T. Duchamp;Zoran Popovic
DOI：
10.1109/iccv.2017.401
发表时间：
2017-03
期刊：
2017 IEEE International Conference on Computer Vision (ICCV)
影响因子：
--
作者：
A. Tewari;M. Zollhöfer;Hyeongwoo Kim;Pablo Garrido;Florian Bernard;P. Pérez;C. Theobalt
通讯作者：
A. Tewari;M. Zollhöfer;Hyeongwoo Kim;Pablo Garrido;Florian Bernard;P. Pérez;C. Theobalt
大规模半稀疏问题的快速非线性最小二乘优化
DOI：
--
发表时间：
2020
期刊：
Computer graphics forum (Print)
影响因子：
--
作者：
M. Fratarcangeli;D. Bradley;Aurel Gruber;Gaspard Zoss;T. Beeler
通讯作者：
T. Beeler