Virtual character performance from speech
Virtual character performance from speech
复制标题
虚拟角色的语音表演
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Ari Shapiro
中科院分区：
文献类型：
--
作者：
S. Marsella;Yuyu Xu;Margot Lhommet;Andrew W. Feng;Stefan Scherer;Ari Shapiro
我们展示了一种方法，用于生成一个3D虚拟人物的表现，从音频信号推断的声学和语义属性的话语。通过对声学信号的韵律分析，我们对重音和音高进行分析，将其与口语相关联并识别激动状态。我们的基于规则的系统执行一个浅的分析话语文本，以确定其语义，语用和修辞的内容。基于这些分析，系统生成面部表情和行为，包括头部运动、眼睛扫视、手势、眨眼和凝视。我们的技术是能够合成的性能，并产生新的手势动画的基础上与其他紧密安排的动画协同。由于我们的方法利用语义除了韵律，我们能够生成虚拟字符的性能，更适合于只使用韵律的方法。我们进行了一项研究，表明我们的技术优于单独使用韵律的方法。
We demonstrate a method for generating a 3D virtual character performance from the audio signal by inferring the acoustic and semantic properties of the utterance. Through a prosodic analysis of the acoustic signal, we perform an analysis for stress and pitch, relate it to the spoken words and identify the agitation state. Our rule-based system performs a shallow analysis of the utterance text to determine its semantic, pragmatic and rhetorical content. Based on these analyses, the system generates facial expressions and behaviors including head movements, eye saccades, gestures, blinks and gazes. Our technique is able to synthesize the performance and generate novel gesture animations based on coarticulation with other closely scheduled animations. Because our method utilizes semantics in addition to prosody, we are able to generate virtual character performances that are more appropriate than methods that use only prosody. We perform a study that shows that our technique outperforms methods that use prosody alone.