Creating Expressive Three-Dimensional Talking Faces

这项工作的总体目标是开发一个栩栩如生的、富有表现力的、会说话的头。这是很难做到的，因为我们对面部特征的细微变化非常敏感。动画序列中的缺陷很容易被发现，并且严重降低了输出的感知质量。对于那些追求视频真实感（与真实视频难以区分）的系统来说尤其如此。之前所有接近视频真实感的方法都是基于图像和二维的；角色的姿势总是正面的，情感通常被忽略，词汇也很有限。这项工作将首次克服所有这些限制。要生成逼真的动画序列，用户只需要提供他们希望动画的句子的文本（或声音）。与此形成鲜明对比的是，皮克斯等动画工作室需要数月（或数年）的时间手动调整动画参数，才能创造出逼真的动画序列。当然，这些序列仅限于电影的脚本-要生成进一步的序列将需要进一步的手动规范参数。该系统将在不需要用户干预的情况下，从有限的可用训练数据中为任意话语生成富有表现力的视觉语音。此外，用户可以指定一个想要的表达式，例如，一个高兴的表达式表示好消息，输出将自动适应该表达式。

The broad aim of this work is to develop a life-like expressive talking head. This is difficult to achieve as we are extremely sensitive to subtle changes in the features of the face. Flaws in animated sequences are easy to detect and severely degrade the perceived quality of the output. This is especially true for systems that strive for videorealism (indistinguishable from real video).All previous approaches that achieve close to videorealism areimage-based and two-dimensional; the pose of the character is always face-on, emotion is usually ignored, and the vocabulary is often limited. This work will overcome, for the first time, all of these limitations.To generate realistic animated sequences, a user need only supply the text (or voice) of the sentence they wish to animate. Contrast this with animation studios, such as Pixar, that require months (or years) of manual tuning of animation parameters to create realistic animated sequences. Of course, these sequences are limited to the script of the movie - to generate further sequences would require further manual specification of the parameters. This system will generate expressive visual speech for any arbitrary utterance from the limited training data available, without the need for user intervention. Also, a user can specify a desired expression, e.g. a happy expression for good news, and the output will automatically be adapted to that expression.