Face transfer with multilinear models
Face transfer with multilinear models
复制标题
DOI：
10.1145/1185657.1185864
复制
发表时间：
2006-07
期刊：
影响因子：
--
通讯作者：
Daniel Vlasic;M. Brand;H. Pfister;J. Popović
中科院分区：
文献类型：
--
作者：
Daniel Vlasic;M. Brand;H. Pfister;J. Popović
面部传输是一种将一个人的视频录制表演映射到另一个人的面部动画的方法。它从单眼视频或电影片段中提取视位（与语音相关的嘴部发音）、表情和三维 (3D) 姿势。然后，这些参数用于生成并驱动目标身份的详细 3D 纹理面部网格，该网格可以无缝渲染回目标镜头中。底层面部模型会自动调整目标的面部表情和嘴型。表演数据可以轻松编辑，改变视位、表情、姿势，甚至目标的身份——属性是单独可控的。这支持各种视频重写和木偶应用程序。面部传输基于 3D 面部网格的多线性模型，该模型可单独参数化由于不同属性（例如身份、表情和视位）而导致的几何变化空间。可分离性意味着这些属性中的每一个都可以独立变化。可以使用统计分析技术从示例的笛卡尔乘积（恒等式 x 表达式 x 视位）来估计多线性模型，但只有在仔细预处理几何数据集以确保一对一对应、最小化交叉耦合伪影并填充任何缺失的示例之后。面部传输为这些问题提供了新的解决方案，并将估计模型与面部跟踪算法联系起来，以提取姿势、表情和视位参数。
Face Transfer is a method for mapping videorecorded performances of one individual to facial animations of another. It extracts visemes (speech-related mouth articulations), expressions, and three-dimensional (3D) pose from monocular video or film footage. These parameters are then used to generate and drive a detailed 3D textured face mesh for a target identity, which can be seamlessly rendered back into target footage. The underlying face model automatically adjusts for how the target performs facial expressions and visemes. The performance data can be easily edited to change the visemes, expressions, pose, or even the identity of the target---the attributes are separably controllable. This supports a wide variety of video rewrite and puppetry applications.Face Transfer is based on a multilinear model of 3D face meshes that separably parameterizes the space of geometric variations due to different attributes (e.g., identity, expression, and viseme). Separability means that each of these attributes can be independently varied. A multilinear model can be estimated from a Cartesian product of examples (identities x expressions x visemes) with techniques from statistical analysis, but only after careful preprocessing of the geometric data set to secure one-to-one correspondence, to minimize cross-coupling artifacts, and to fill in any missing examples. Face Transfer offers new solutions to these problems and links the estimated model with a face-tracking algorithm to extract pose, expression, and viseme parameters.