Neural mechanisms of perceiving dynamic real-world environments

人们被一个复杂的动态世界所包围，这个世界以众多的景象和声音为特征。例如，在树林里徒步旅行时，他们可能会看到前面灌木丛中勾勒出的小路，或者他们可能会发现一些鹿；他们可能会听到附近小溪的潺潺声或树梢在风中摇曳的声音。另一方面，从有轨电车上走到繁忙的城市街道上，人们可能会看到高楼大厦、人行道上的行人或街上的汽车；他们可能会听到远处的发动机噪音或警笛。特定的环境有助于塑造人们的行为，比如导航、躲避威胁、视觉搜索或记忆编码。我的长期研究计划的核心是探索大脑在何处以及如何处理身处复杂、动态环境中的体验，以及哪些刺激特性传达了这种体验。

People are surrounded by a complex dynamic world characterized by a multitude of sights and sounds. When hiking in the woods, for instance, they may see the path outlined in the undergrowth ahead, or perhaps they may spot some deer; they may hear a stream purling nearby or the treetops swaying in the wind. Stepping out of a streetcar onto a busy city street, on the other hand, people may see tall buildings, pedestrians on the sidewalks or cars in the street; they may hear engine noises or a police siren in the distance. The particular environment contributes to shaping people’s behaviour for tasks such as navigation, threat avoidance, visual search or memory encoding. At the heart of my long-term research program is the exploration of where and how the experience of being in a complex, dynamic environment is processed in the brain, and which stimulus properties convey this experience. Determining the sensory features that enable scene perception in vision and hearing will be an important first step. However, finding the neural representations and mechanisms that underlie real-world perception across sensory modalities will be crucial for understanding the perception of being in a particular environment, e.g., the experience of “being in a forest.” I propose a research program geared toward deciphering the neural mechanisms of audiovisual perception of dynamic real-world scenes: 1. Collect, annotate and publish a database of real-world audiovisual scenes; 2. Determine the speed and accuracy with which people identify specific environments in behavioral experiments; 3. Measure and decode neural activity patterns of participants’ scene perception using time-resolved functional magnetic resonance imaging (fMRI); 4. Construct a computational model of audiovisual scene perception and test the model with ambiguous scenes and transitions at natural boundaries between environments. Over the next five years I will train at least two graduate students and approximately 14 undergraduate students in skills crucial for the project, ranging from fMRI scanning and human psychophysics to computational modeling and scientific writing. In addition to advancing our scientific knowledge, the results of this project have the potential to improve our understanding of the perceptual reality encountered by people with visual or auditory impairments. Furthermore, uncovering the computational mechanisms of environmental perception has important applications in engineering. Algorithms derived from this research could lead to improved context awareness in engineering products, such as cars, cell phones or assistive robots, allowing them to adapt their functionality to their environment similar to humans. To conclude, the proposed research has high potential to contribute to Canada’s knowledge-based economy, both through technological advances and through the training of highly qualified personnel.