Creating Expressive Three-Dimensional Talking Faces

这项工作的主要目标是开发一个栩栩如生的表达说话的头。这是很难实现的，因为我们对面部特征的细微变化非常敏感。动画序列中的缺陷很容易检测到，并且会严重降低输出的感知质量。这对于追求视频真实感（与真实的视频难以区分）的系统来说尤其如此。以前所有接近视频真实感的方法都是基于图像和二维的;角色的姿势总是正面的，情感通常被忽略，词汇量通常有限。这项工作将首次克服所有这些限制。为了生成逼真的动画序列，用户只需要提供他们希望动画的句子的文本（或声音）。这与动画工作室（如Pixar）形成对比，后者需要数月（或数年）手动调整动画参数才能创建逼真的动画序列。当然，这些序列仅限于电影的脚本-生成进一步的序列将需要进一步手动指定参数。该系统将从有限的训练数据中为任意话语生成表达性视觉语音，而不需要用户干预。此外，用户可以指定期望的表达，例如，对于好消息的高兴的表达，并且输出将自动适应该表达。

The broad aim of this work is to develop a life-like expressive talking head. This is difficult to achieve as we are extremely sensitive to subtle changes in the features of the face. Flaws in animated sequences are easy to detect and severely degrade the perceived quality of the output. This is especially true for systems that strive for videorealism (indistinguishable from real video).All previous approaches that achieve close to videorealism areimage-based and two-dimensional; the pose of the character is always face-on, emotion is usually ignored, and the vocabulary is often limited. This work will overcome, for the first time, all of these limitations.To generate realistic animated sequences, a user need only supply the text (or voice) of the sentence they wish to animate. Contrast this with animation studios, such as Pixar, that require months (or years) of manual tuning of animation parameters to create realistic animated sequences. Of course, these sequences are limited to the script of the movie - to generate further sequences would require further manual specification of the parameters. This system will generate expressive visual speech for any arbitrary utterance from the limited training data available, without the need for user intervention. Also, a user can specify a desired expression, e.g. a happy expression for good news, and the output will automatically be adapted to that expression.