Where is "it"? Event Synchronization in Gaze-Speech Input Systems
Where is "it"? Event Synchronization in Gaze-Speech Input Systems
复制标题
它在哪里”？
DOI：
--
复制
发表时间：
2003
期刊：
影响因子：
--
通讯作者：
Chandra Sekhar Mantravadi
中科院分区：
文献类型：
--
作者：
Manpreet Kaur;M. Tremaine;Ning Huang;J. Wilder;Z. Gacovski;F. Flippo;Chandra Sekhar Mantravadi
凝视和语言之间的关系是为了在计算机屏幕上将物体从一个位置移动到另一个位置的简单任务而探索的。受试者将指定的对象从一组对象移动到屏幕上的新位置，并说“移动到那里”。目光和语音数据被捕获，以确定我们是否可以稳健地预测所选物体和目的地位置。我们发现，最接近期望对象的源固定开始，有很大的概率，在单词“移动”开始之前。对讲话开始前后所有注视的分析表明，识别要移动的物体的最佳注视平均发生在讲话开始前630毫秒，个别受试者的注视范围为150至1200毫秒。在这些时间里，个体之间的差异相对较小，尽管不同受试者之间的差异很大。选择最靠近单词“Move”开头的固定点作为要移动的对象的指示器，使所有受试者的系统准确率接近95%。因此，尽管受试者之间存在显著差异，但我们相信语音和凝视整合模式可以可靠地为个体用户建模，从而用于提高多模态系统的性能。
The relationship between gaze and speech is explored for the simple task of moving an object from one location to another on a computer screen. The subject moves a designated object from a group of objects to a new location on the screen by stating, "Move it there". Gaze and speech data are captured to determine if we can robustly predict the selected object and destination position. We have found that the source fixation closest to the desired object begins, with high probability, before the beginning of the word "Move". An analysis of all fixations before and after speech onset time shows that the fixation that best identifies the object to be moved occurs, on average, 630 milliseconds before speech onset with a range of 150 to 1200 milliseconds for individual subjects. The variance in these times for individuals is relatively small although the variance across subjects is large. Selecting a fixation closest to the onset of the word "Move" as the designator of the object to be moved gives a system accuracy close to 95% for all subjects. Thus, although significant differences exist between subjects, we believe that the speech and gaze integration patterns can be modeled reliably for individual users and therefore be used to improve the performance of multimodal systems.
对于简短的演示来说，同时进行比顺序进行更好。
DOI：
10.1364/josaa.12.000441
发表时间：
1995
期刊：
Journal of the Optical Society of America. A, Optics, image science, and vision
影响因子：
--
作者：
Hung,GK;Wilder,J;Curry,R;Julesz,B
通讯作者：
Julesz,B