Combining active vision with multimodal speech comprehension (C05)

主动视觉与多模态语音理解相结合（C05）

• 批准号: 289664153
• 金额: --
• 依托单位: Arbeitsbereich Natürlichsprachliche Systeme (NATS)
• 依托单位国家: 德国
• 项目类别: CRC/Transregios
• 财政年份: 2016
• 资助国家: 德国
• 起止时间: 2015-12-31 至 2018-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/289664153?language=en
• 关键词: Combining; active; vision; multimodal; speech

Project C5 uses eye tracking to investigate how humans integrate visual clues from the environment while they simultaneously hear speech that refers to that environment. The goal is to use insights from this investigation not only to improve existing computational models of visually informed language comprehension, but also to increase the effectiveness of crossmodal learning in humans. Experiments will explore how eye movements are modulated by cognitive activity and influenced by culture.

Project C5使用眼动追踪来研究人类如何在听到涉及该环境的语音的同时整合来自环境的视觉线索。我们的目标是利用这项调查的见解，不仅要改善现有的视觉语言理解的计算模型，而且要提高人类跨模态学习的有效性。实验将探索眼动是如何被认知活动调节和文化的影响。