Cognitive Knowledge Structures in Scene Perception - Behavioral and Neural Correlates of Semantic and Syntactic Scene Processing

虽然我们的视觉世界是复杂的，但它的构成遵循一定的规则。例如，大多数对象需要一个水平面才能放置，而两个对象很少共存于同一位置。即使除了这些关于物体的物理属性的基本知识之外，我们似乎也有关于在某些场景中往往会发现什么物体的详细知识。这项研究计划的目的是进一步了解关于场景中物体的认知知识结构和期望如何帮助我们在观看和互动自然场景时有效地引导我们的注意力和行动。拟议的研究有三个主要目标：首先，该计划将致力于识别并利用支配场景构成的规则。为此，我们将对图像统计数据进行大规模分析，以创建“真实世界词典”，这是一个易于访问的数据库，其中包含有关场景内和场景之间的对象频率和典型位置的信息。此外，我们将创建一个高度受控的图像素材语料库，用于实验用途。该计划的第二个目标是关于场景知识的发展。类似于语言处理中的语法，它允许我们理解无限数量的新句子，我们似乎拥有一种即使在不熟悉的地方也能指导我们的“场景语法”。通过研究成人的人工规则学习和婴幼儿场景理解的发展，我们希望找出这种场景知识是如何随着时间的推移而发展的。受规则支配的预期的出现将通过心理物理学、眼球跟踪和大脑反应的测量相结合来跟踪。该研究计划的第三个目标涉及在积极探索真实世界的3D场景时使用场景知识。参与者将戴着移动的头盔眼球追踪器搜索物体并与之互动。这一设置将允许我们测试，一旦搜索超越2D显示，实际上涉及到现实世界中的主动身体运动，哪些指导来源将成为主导。此外，该计划的一个重要部分将包括评估、进一步开发和使用新兴技术，如眼睛运动和大脑反应的联合登记，以测量所谓的注视相关电位(FRP)。特别是在观看场景期间，根据观察者当前正在观看的内容在线测量大脑潜力，将使我们能够更好地将认知过程与每个观察者采样的实际视觉输入联系起来。这一计划的成果在科学内外的许多应用中都具有巨大的潜力，从更具认知启发的技术系统到婴幼儿发育障碍的早期诊断。

While our visual world is complex, its composition conforms to certain rules. For instance, most objects require a horizontal surface to rest upon and two objects seldom coexist in the very same location. Even beyond such basic knowledge about physical properties of objects, we seem to have detailed knowledge regarding what objects tend to be found where within certain scenes. The aim of this research program is to gain further insight into how cognitive knowledge structures and expectations regarding objects in scenes help us to efficiently guide our attention and action when viewing and interacting within naturalistic scenes.The proposed research has three major objectives: First, the program will be dedicated to identifying and then utilizing the rules that govern the composition of scenes. For this purpose, we will perform large-scale analyses of image statistics to create the “Real-World Lexicon” — an easily accessible database containing information regarding object frequencies and typical locations within and across scenes. In addition, we will create a corpus of highly controlled image material for experimental use. The second aim of the program concerns the development of scene knowledge. Similar to the grammar in language processing that allows us to understand an unlimited number of novel sentences, we seem to possess a “scene grammar” that guides us even in unfamiliar places. By studying both artificial rule learning in adults and the development of scene understanding in infants and toddlers, we hope to find out how such scene knowledge develops over time. The emergence of rule-governed expectations will be tracked by a combination of psychophysics, eye tracking, and the measurement of brain responses. The third objective of the research program concerns the use of scene knowledge during active exploration of real-world, 3D scenarios. Participants will search for and interact with objects, while wearing a mobile, headmounted eye tracker. This setup will allow us to test what sources of guidance become dominant once search takes place beyond 2D displays, and actually involves active body movements in the real world.Moreover, an essential part of the program will include the evaluation, further development, and use of emerging techniques, such as the co-registration of eye movements and brain responses to measure so-called fixation-related potentials (FRPs). Particularly during scene viewing, the online measurement of brain potentials in response to what an observer is currently looking at will allow us to better relate cognitive processes to the actual visual input sampled by each individual observer. The outcome of this program has great potential for many applications within and outside of science ranging from more cognitively inspired technical systems to early diagnosis of developmental disorders in infants and toddlers.