Developing a cross-methodlogical processing account of situated language comprehension

研究表明，不仅语言知识和世界知识，而且场景信息也会影响渐进式理解。具体而言，三个实验范式的理解和视觉过程如何相互作用的contributedinsights：在spokencomprehension的场景中的眼动跟踪，测量的反应延迟在图片验证任务，和电生理记录在情境中。句子理解的心理语言学理论，然而，在提供语言知识整合的详细说明的同时，还没有充分地解决理解与视觉过程之间的相互作用。先前自己的工作已经初步描述了语言和视觉过程之间的协调相互作用，基于口语理解过程中场景中的眼动跟踪发现。拟议的研究将发展一个更完整的解释，通过系统地利用所有三种范式提供的调查结果和措施，通过这种方式，我们可以同时利用关于视觉注意、加工复杂性和潜在神经过程的证据来完善我们的解释。不同的方法将进一步支持跨书面和口头语言模式以及不同的视觉环境（简单与复杂，静态与动态）扩展帐户。

It has been shown that not only linguistic and world knowledge, but also scene information can influence incremental comprehension. Specifically, three experimental paradigms have contributedinsights into how comprehension and visual processes interact: eye tracking in scenes during spokencomprehension, the measuring of response latencies in sentence-picture verification tasks, and electrophysiological recordings in situated contexts. Psycholinguistic theories of sentence comprehension, however - while providing detailed accounts of the integration of linguistic knowledge- do not yet adequately address the interaction of comprehension with visual processes.Prior own work has sketched a preliminary account of the coordinated interplay between linguisticand visual processes based on findings from eye tracking in scenes during spoken comprehension.The proposed research will develop a more complete account of situated scene sentencecomprehension through the systematic exploitation of findings and measures provided by all three paradigms. In this way, evidence concerning visual attention, processing complexity, and underlying neural processes can be drawn upon simultaneously in refining our account. The diverse methods will further support extending the account across written and spoken language modalities, as well as for different visual contexts (simple vs. complex, static vs. dynamic).