Between linguistic attention and gaze fixations inmultimodal conversational interfaces
Between linguistic attention and gaze fixations inmultimodal conversational interfaces
复制标题
多模态会话界面中的语言注意和注视之间的关系
DOI：
10.1145/1647314.1647339
复制
发表时间：
2009
影响因子：
11.2
通讯作者：
F. Ferreira
中科院分区：
文献类型：
--
作者：
Rui Fang;J. Chai;F. Ferreira
在多模态人机对话中，成功地解释人类的注意力是至关重要的。虽然注意在语言处理和视觉处理中已经得到了广泛的研究，但在多模态会话界面中语言注意与视觉注意的关系尚不清楚。为了解决这个问题，我们进行了初步的调查，在人机对话过程中，语言话语所反映的注意力与凝视注视所指示的注意力如何一致。我们的实证研究结果表明，更多的关注实体的语言话语的基础上对应于更高的注视强度。语言过渡越平滑，相应注视分布之间的距离越小。这些发现提供了关于如何将语言和凝视结合起来预测注意力的见解，这在许多任务中具有重要意义，如单词习得和物体识别。
In multimodal human machine conversation, successfully interpreting human attention is critical. While attention has been studied extensively in linguistic processing and visual processing, it is not clear how linguistic attention is aligned with visual attention in multimodal conversational interfaces. To address this issue, we conducted a preliminary investigation on how attention reflected by linguistic discourse aligns with attention indicated by gaze fixations during human machine conversation. Our empirical findings have shown that more attended entities based on linguistic discourse correspond to higher intensity of gaze fixations. The smoother a linguistic transition is, the less distance between corresponding fixation distributions. These findings provide insight into how language and gaze can be combined to predict attention, which have important implications in many tasks such as word acquisition and object recognition.