Neural mechanisms of perceiving dynamic real-world environments

人们被一个复杂的动态世界所包围，这个世界的特点是各种各样的景象和声音。例如，当他们在森林中徒步旅行时，他们可能会看到前面灌木丛中勾勒出的小路，或者他们可能会看到一些鹿，他们可能会听到附近的小溪发出咕噜声，或者树梢在风中摇曳。另一方面，走出有轨电车进入繁忙的城市街道，人们可能会看到高楼、人行道上的行人或街上的汽车；他们可能会听到发动机的噪音或远处的警笛。特定的环境有助于塑造人们在导航、威胁回避、视觉搜索或记忆编码等任务中的行为。我长期研究计划的核心是探索处于复杂、动态环境中的经历是在哪里以及如何在大脑中处理的，以及哪些刺激特性传达了这种体验。 确定能够在视觉和听觉上实现场景感知的感觉特征将是重要的第一步。然而，找到跨越感觉通道的真实世界感知的神经表示和机制对于理解在特定环境中的感知至关重要，例如，在森林中的体验。我提出了一个旨在破译动态真实场景视听感知神经机制的研究计划：1.收集、注释和发布真实世界视听场景的数据库；2.在行为实验中确定人们识别特定环境的速度和准确性；3.使用时间分辨功能磁共振成像(FMRI)测量和解码参与者场景感知的神经活动模式；4.构建视听场景感知的计算模型，并在场景模糊和环境之间的自然边界处进行转换的情况下对该模型进行测试。 在接下来的五年里，我将培训至少两名研究生和大约14名本科生掌握对该项目至关重要的技能，范围从功能磁共振成像扫描和人类心理物理学到计算建模和科学写作。除了提高我们的科学知识，这个项目的结果还有可能提高我们对视觉或听觉障碍患者所遇到的感知现实的理解。此外，揭示环境感知的计算机制在工程上也有重要的应用。这项研究得出的算法可能会提高汽车、手机或辅助机器人等工程产品的上下文感知能力，使它们能够像人类一样调整自己的功能以适应环境。总之，拟议的研究具有很大潜力，可通过技术进步和培训高素质人员为加拿大的知识经济作出贡献。

People are surrounded by a complex dynamic world characterized by a multitude of sights and sounds. When hiking in the woods, for instance, they may see the path outlined in the undergrowth ahead, or perhaps they may spot some deer; they may hear a stream purling nearby or the treetops swaying in the wind. Stepping out of a streetcar onto a busy city street, on the other hand, people may see tall buildings, pedestrians on the sidewalks or cars in the street; they may hear engine noises or a police siren in the distance. The particular environment contributes to shaping people’s behaviour for tasks such as navigation, threat avoidance, visual search or memory encoding. At the heart of my long-term research program is the exploration of where and how the experience of being in a complex, dynamic environment is processed in the brain, and which stimulus properties convey this experience. Determining the sensory features that enable scene perception in vision and hearing will be an important first step. However, finding the neural representations and mechanisms that underlie real-world perception across sensory modalities will be crucial for understanding the perception of being in a particular environment, e.g., the experience of “being in a forest.” I propose a research program geared toward deciphering the neural mechanisms of audiovisual perception of dynamic real-world scenes: 1. Collect, annotate and publish a database of real-world audiovisual scenes; 2. Determine the speed and accuracy with which people identify specific environments in behavioral experiments; 3. Measure and decode neural activity patterns of participants’ scene perception using time-resolved functional magnetic resonance imaging (fMRI); 4. Construct a computational model of audiovisual scene perception and test the model with ambiguous scenes and transitions at natural boundaries between environments. Over the next five years I will train at least two graduate students and approximately 14 undergraduate students in skills crucial for the project, ranging from fMRI scanning and human psychophysics to computational modeling and scientific writing. In addition to advancing our scientific knowledge, the results of this project have the potential to improve our understanding of the perceptual reality encountered by people with visual or auditory impairments. Furthermore, uncovering the computational mechanisms of environmental perception has important applications in engineering. Algorithms derived from this research could lead to improved context awareness in engineering products, such as cars, cell phones or assistive robots, allowing them to adapt their functionality to their environment similar to humans. To conclude, the proposed research has high potential to contribute to Canada’s knowledge-based economy, both through technological advances and through the training of highly qualified personnel.