Modeling Multimodal Behaviors from Speech Prosody
Modeling Multimodal Behaviors from Speech Prosody
复制标题
从语音韵律建模多模态行为
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
T. Artières
中科院分区：
文献类型：
--
作者：
Yu Ding;C. Pelachaud;T. Artières
头部和眉毛的动作是重要的沟通手段。它们与语音韵律高度同步。赋予虚拟代理同步的言语和非言语行为可以增强他们的沟通表现。在本文中，我们提出了一种基于连接语音韵律和面部运动的统计模型的虚拟代理动画模型。首先提出了完全参数化的隐马尔可夫模型，以捕获从视频语料库中提取的人脸的语音和面部运动之间的紧密关系，然后从语音信号自动驱动虚拟代理的行为。在模型构建过程中还考虑了头部和眉毛运动之间的相关性。进行了主观和客观评估来验证该模型。
Head and eyebrow movements are an important communication mean. They are highly synchronized with speech prosody. Endowing virtual agent with synchronized verbal and nonverbal behavior enhances their communicative performance. In this paper, we propose an animation model for the virtual agent based on a statistical model linking speech prosody and facial movement. A fully parameterized Hidden Markov Model is proposed first to capture the tight relationship between speech and facial movement of a human face extracted from a video corpus and then to drive automatically virtual agent’s behaviors from speech signals. The correlation between head and eyebrow movements is also taken into account during the building of the model. Subjective and objective evaluations were conducted to validate this model.