Collaborative Proposal: Object and Action Recognition in Time Sequences of Images: Computational Neuroscience and Neurophysiology

摘要正常的视觉不是静态的：时间是我们看到的自然世界的一个关键维度。最终理解生物视觉需要理解用于识别物体和动作的神经机制。因此，本研究的重点是研究灵长类动物的视觉系统如何在图像的时间序列中识别物体和动作。该项目的一个元目标是利用计算方法和生理实验之间的协同作用，从而更好地了解大脑功能，同时开发更好的计算机视觉算法。图像时间序列中的物体识别对识别系统提出了重大挑战，因为它既需要对形状的选择性，又需要对外观变化的不变性。该项目将通过在其模型神经元中添加时间动态和处理视频序列的能力来扩展现有的腹侧流计算模型。它还将扩展背侧流的工作模型，以了解它和腹侧流在动态视觉识别中所起的相对作用。同时，将对包括IT和STS区域在内的高水平视觉区域的单个单元和多个单个单元进行记录，以表征单个神经元对特定图像序列的形状动态的调整。通过将建模和生理学相结合，这项工作将寻找一种计算解释，来解释视觉皮层的高级区域如何随着时间的推移识别物体和动作，以及它们如何学习。这种专注于动态感知信息处理的综合努力，除了直接指导计算机视觉的建模和工程工作外，还可以对当前关于自闭症、阅读障碍和中风影响的理论产生重大而直接的影响。拟议的研究与教育和教学紧密结合，研究中使用的资源，包括视频数据库、视觉刺激、建模软件和实验数据，将向广泛的科学界提供。有关该项目的信息及其进展情况可在http://cbcl.mit.edu/projects/NSF-CRCNS/index.html上查阅

Last Modified Date: 08/01/08 Last Modified By: Daniel F. DeMenthon Abstract Normal vision is not static: time is a key dimension of the natural world we see. The eventual understanding of biological vision requires understanding the neural mechanisms used to recognize objects and actions over time. Thus the focus of the proposed research is to study how the primate visual system recognizes objects and actions in time sequences of images. A meta-goal of this project is to exploit the synergies between computational approaches and physiological experiments to lead to a better understanding of brain function and at the same time to develop better computer vision algorithms. Object recognition in time sequences of images presents a significant challenge for recognition systems, because it requires both selectivity to shape and invariance to changes of appearance in time.. This project will extend an existing computational model of the ventral stream by adding temporal dynamics in its model neurons and the ability to process video sequences. It will also expand a working model of the dorsal stream to understand the relative roles that it and the ventral stream play in dynamic visual recognition. At the same time, recordings from single units, and multiple single units, from high level visual areas including IT and regions of the STS will be made in order to characterize the tuning of single neurons to the shape dynamics of specific image sequences. By combining modeling and physiology, this work will search for a computational explanation for how the higher areas of the visual cortex recognize objects and actions over time and how they can learn. This integrative effort, which is focused on processing of dynamic perceptual information, can have a significant and direct impact on current theories of autism, dyslexia, and effects of stroke, in addition to directly guiding modeling and engineering efforts in computer vision. The proposed research is tightly coupled to education and teaching, and resources used in the research, including databases of videos, visual stimuli, the modeling software and the experimental data will be made available to the broad scientific community. Information on the project and its progress will be available at http://cbcl.mit.edu/projects/NSF-CRCNS/index.html