Trading Spaces: How Humans and Humanoids Use Speech and Gesture to Give Directions
Trading Spaces: How Humans and Humanoids Use Speech and Gesture to Give Directions
复制标题
交易空间：人类和类人动物如何使用语音和手势来指示方向
DOI：
10.1002/9780470512470.ch8
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
Justine Cassell
中科院分区：
文献类型：
--
作者：
S. Kopp;Paul Tepper;Kim Ferriman;Kristina Striegnitz;Justine Cassell
人类凭直觉用手势来指导方向。这些手势已被证明与图表和指导语言具有相同的基本概念结构，但问题在于它们如何进行交流，因为它们的形式没有被编码，而且实际上可能因人或情况而异。基于
基于对指导中的语言和手势的研究，我们描述了一个框架，用于将手势图像分析为语义单元（图像描述特征），并将这些单元与形态特征（手形、轨迹等）联系起来。这种基于特征的框架允许实现多模式方向的集成微规划器，直接从交流目标中导出自然语言和手势的形式。使用这个微规划器，我们开发了一个具体的对话代理，它可以在与真人的指导对话中执行适当的语音和新颖的手势。
Humans intuitively accompany direction-giving with gestures. These gestures have been shown to have the same underlying conceptual structure as diagrams and direction-giving language, but the puzzle is how they communicate given that their form is not codified, and may in fact differ from one person or situation to the next. Based on
results from a study on language and gesture in direction-giving, we describe a framework to analyze gestural images into semantic units (image description features), and to link these units to morphological features (hand shape, trajectory, etc.). This feature-based framework allows for implementing an integrated microplanner for multimodal directions that derives the form of both natural language and gesture directly from communicative goals. Using this microplanner we developed an embodied conversational agent that can perform appropriate speech and novel gestures in direction-giving conversation with real humans.