Multimodal expression in virtual humans
Multimodal expression in virtual humans
复制标题
虚拟人的多模态表达
DOI：
10.1002/cav.127
复制
发表时间：
2006
影响因子：
1.1
通讯作者：
Ana Paiva
中科院分区：
文献类型：
--
作者：
C. D. Melo;Ana Paiva
提出了一种真实的实时虚拟人多模态表情模型。五种模式探索身体的启示：确定性，非确定性，手势，面部和声音表达。确定性表达式是关键帧身体动画。非确定性表达是基于机器人的程序化身体动画。声音的表达是声音合成，通过节日，和参数化，通过SABLE。面部表情是通过参数化的基于肌肉的面部模型进行的对口型和情感表达。受心理语言学的启发，手势表达是一种非常规的、特殊的、无意识的手势动画，描述为葡萄牙手语手形、位置和方向的序列。受艺术的启发，一种形式超越了身体，探索环境的启示，并通过相机，灯光和音乐表达情感。为了控制多模态表达，这项工作提出了一种高级集成同步标记语言-表达标记语言。最后，三项研究，共涉及197个主题，评估了该模型在讲故事的背景下，并产生了可喜的结果。版权所有© 2006约翰威利父子有限公司。
This work proposes a real‐time virtual human multimodal expression model. Five modalities explore the affordances of the body: deterministic, non‐deterministic, gesticulation, facial, and vocal expression. Deterministic expression is keyframe body animation. Non‐deterministic expression is robotics‐based procedural body animation. Vocal expression is voice synthesis, through Festival, and parameterization, through SABLE. Facial expression is lip‐synch and emotion expression through a parametric muscle‐based face model. Inspired by psycholinguistics, gesticulation expression is unconventional, idiosyncratic, and unconscious hand gestures animation described as sequences of Portuguese Sign Language hand shapes, positions and orientations. Inspired by the arts, one modality goes beyond the body to explore the affordances of the environment and express emotions through camera, lights, and music. To control multimodal expression, this work proposes a high‐level integrated synchronized markup language—expressive markup language. Finally, three studies, involving a total of 197 subjects, evaluated the model in storytelling contexts and produced promising results. Copyright © 2006 John Wiley & Sons, Ltd.