Neural Basis of Active Perception in Natural Environment

了解动物如何感知复杂的自然环境并对其采取行动是神经科学中最紧迫的挑战之一，其应用不仅有可能彻底改变我们对大脑的理解，还可能彻底改变机器视觉、人工智能和机器人技术。到目前为止，研究主动视觉的神经基础——视觉刺激如何在不同的任务条件下引起眼球运动——很大程度上局限于简化的实验室刺激，给过度训练的动物执行刻板的任务。在国家科学基金会、加拿大健康研究所和国家地理空间情报局的资助下，加拿大皇后大学的Douglas Munoz博士和南加州大学的Laurent Itti博士将结合神经生理学和计算模型来研究自然环境下的自由观看。使用多电极阵列，该项目将记录在一个深层的大脑结构，称为上丘（SC）。SC是一个分层结构，包括几个很容易理解的神经图谱，从浅层的纯粹感觉表征到深层与控制眼球运动有关的感觉运动表征。该项目将从描述简单刺激条件下SC中神经元的反应开始：当动物只是看着显示器上的中央固定十字，而在其他视觉位置呈现小的孤立图案时；当动物在一系列分散注意力的物品中搜索一个奇怪的物品时；当动物检查自然图像和视频剪辑时。该项目将扩展研究者的突出图理论和模型，并开发一个新的SC模型。完整的模型将从任何图像或视频剪辑中预测哪些视觉位置更突出，与任务相关，以及眼球运动的候选目标。该项目利用了神经生理学实验室（联合PI Douglas P. Munoz）和计算建模实验室（PI Laurent Itti）之间的跨学科合作。这将允许，通过实验和建模的结合，解释在自然观测中收集的大量数据，否则无法解读。相反，这些理论将指导进一步的实验。以前从未尝试过在自由观看自然视频期间将多单元记录与建模相结合，并且预计它将导致对自然条件下感知如何映射为行动的新理解。该项目将支持本科生、研究生和博士后研究人员，他们将受益于生理学和计算技术的结合，以及研究人员的教学。除出版物外，所有理论和算法源代码将免费分发，数据将通过CRCNS数据共享网站提供。因此，这项研究有望在理解动物如何使用视觉信息来指导行为，以及如何在面对复杂的自然世界时构建以类似方式行事的机器方面取得新的、广泛的基础进展。

Understanding how animals perceive and act upon complex natural environments is one of the most pressing challenges in neuroscience, with applications that have potential to revolutionize not only our understanding of the brain, but also machine vision, artificial intelligence, and robotics. Until now, studying the neural basis of active vision - how visual stimuli give rise to eye movements under diverse task conditions - has largely been restricted to simplified laboratory stimuli, presented to overtrained animals performing stereotypical tasks. With funding from the National Science Foundation, the Canadian Institute of Health Research, and the National Geospatial Intelligence Agency, Dr. Douglas Munoz at Queens University in Canada and Dr. Laurent Itti at the University of Southern California will combine neurophysiology and computational modeling to investigate free viewing in natural environments. Using multi-electrode arrays, this project will record in a deep brain structure, called the superior colliculus (SC). The SC is a layered structure comprising several well-understood neural maps, from purely sensory representations in the superficial layers, to sensorimotor representations linked to the control of eye movements in the deeper layers. The project will start by characterizing responses of neurons in the SC under simple stimulus conditions: When the animal is simply looking at a central fixation cross on a display while small isolated patterns are presented at other visual locations; when the animal searches for an oddball item among an array of distracting items; and when the animal inspects natural images and video clips. The project will extend the investigators' salience map theories and models, and develop a new model of the SC. The complete model will predict, from any image or video clip, which visual locations are more salient, task-relevant, and candidate targets for eye movements.The project leverages a cross-disciplinary collaboration between a neurophysiology lab (co-PI Douglas P. Munoz) and a computational modeling lab (PI Laurent Itti). This will allow, through the combination of experiments and modeling, the interpretation of an otherwise undecipherable mass of data collected during natural viewing. Conversely, the theories will guide further experiments. Coupling multi-unit recording with modeling during free-viewing of natural videos has never been attempted before, and it is expected that it will lead to new understanding of how percepts map into actions under natural conditions. The project will support undergraduate and graduate students, and post-doctoral researchers, who will benefit from exposure to combined physiological and computational techniques, as will the investigators' teaching. In addition to publications, all theory and algorithm source code will be freely distributed, and data will be available through the CRCNS data sharing web site. This research is hence expected to lead to new and broadly accessible fundamental advances in the understanding of how animals use visual information to guide behavior, and how one could build machines which act in similar ways when faced with the complex natural world.