REU: CONACyT: Speech Driven Facial Animation

这个项目将产生一个忠实的3D动画的一个扬声器的脸，准确地清晰可见的语音，并保持说话者的话语和相应的面部运动之间的自然同步，包括尽可能多的手势韵律。 这是通过从语音输入到显示器的一系列变换来实现的。 从语音学的角度来看，PI将在音素以下的粒度级别上研究音素和视位之间的关系。 从计算机图形学的角度来看，他们将模拟面部特征，使现实的同步与话语发生使用计算机视觉技术的活动轮廓和动画技术的多层隐式模型。 最后，从韵律学的角度来看，他们将应用手势分数的概念，这应该占，在一个更原则的基础上，为语音中的韵律现象和语境的影响。 手势模型已经发展成为一种新的任务动态方法不变的发音手势的基础上的编排（通过一个相当吝啬的“分数”）的二阶弹簧质量系统达到目标的具体目标。 声学特征表包括表征手势分数的定相和持续时间。 这项工作不仅对动画（电影和电视，听力受损，低带宽电话，可重复的韵律界面），而且对跨两种语言（英语和西班牙语）的韵律的基本语言建模具有重要意义。

This project will produce a faithful 3D animation of the face of a speaker, accurately articulating visible speech, and preserving the natural synchronism between the utterances of the speaker and the corresponding facial movements, including as much of the gestural prosody as possible. This is accomplished through a series of transformations from the speech input to the display. From the phonetic perspective the PIs will study the relation between phonemes and visemes at a level of granularity below the phoneme. From the computer graphics perspective, they will model the facial features so that realistic synchronism with the utterances takes place using the computer vision technique of active contours and the animation technique of multilayer implicit models. Finally, from a prosodic point of view they will apply the concepts of gestural scores which should account, on a more principled basis, for the prosodic phenomena and contextual influences in speech. The gestural model has evolved into a novel task-dynamic approach to invariant articulatory gestures based on an orchestration (by means of a rather parsimonious "score") of a second order spring-mass system reaching for target specific goals. An acoustic features table includes the phasing and duration which characterize gestural scores. This work has important implications not only for animation (films and television, interfaces for the hearing impaired, low bandwidth telephony, repeatable prosody), but also for basic linguistic modeling of prosody across the two languages (English and Spanish) being considered.