Collaborative Proposal: Object and Action Recognition in Time Sequences of Images: Computational Neuroscience and Neurophysiology

最后修改日期：8/01/08最后修改人：丹尼尔·F·德门森摘要正常视觉不是静态的：时间是我们所看到的自然世界的一个关键维度。对生物视觉的最终理解需要理解随着时间的推移用来识别物体和动作的神经机制。因此，本研究的重点是研究灵长类视觉系统如何识别图像时间序列中的物体和动作。这个项目的一个元目标是利用计算方法和生理实验之间的协同作用，以更好地理解大脑功能，同时开发更好的计算机视觉算法。图像时间序列中的目标识别对识别系统来说是一个巨大的挑战，因为它既需要形状的选择性，也需要对外观随时间变化的不变性。该项目将通过在其模型神经元中添加时间动力学和处理视频序列的能力来扩展现有的腹侧流计算模型。它还将扩展背侧流的工作模型，以了解它和腹侧流在动态视觉识别中扮演的相对角色。同时，将从包括IT和STS区域在内的高级视觉区域进行单个单元和多个单个单元的记录，以表征单个神经元对特定图像序列的形状动态的调节。通过将建模和生理学结合起来，这项工作将寻找一种计算解释，解释视觉皮质的高级区域如何随着时间的推移识别对象和动作，以及它们如何学习。这种集中于动态感知信息处理的综合努力，除了直接指导计算机视觉中的建模和工程工作外，还可以对当前关于自闭症、阅读障碍和中风影响的理论产生重大和直接的影响。拟议的研究与教育和教学密切相关，研究中使用的资源，包括视频、视觉刺激、建模软件和实验数据的数据库，将向广大科学界提供。有关该项目及其进展的信息将在http://cbcl.mit.edu/projects/NSF-CRCNS/index.html上获得。

Last Modified Date: 08/01/08 Last Modified By: Daniel F. DeMenthon Abstract Normal vision is not static: time is a key dimension of the natural world we see. The eventual understanding of biological vision requires understanding the neural mechanisms used to recognize objects and actions over time. Thus the focus of the proposed research is to study how the primate visual system recognizes objects and actions in time sequences of images. A meta-goal of this project is to exploit the synergies between computational approaches and physiological experiments to lead to a better understanding of brain function and at the same time to develop better computer vision algorithms. Object recognition in time sequences of images presents a significant challenge for recognition systems, because it requires both selectivity to shape and invariance to changes of appearance in time.. This project will extend an existing computational model of the ventral stream by adding temporal dynamics in its model neurons and the ability to process video sequences. It will also expand a working model of the dorsal stream to understand the relative roles that it and the ventral stream play in dynamic visual recognition. At the same time, recordings from single units, and multiple single units, from high level visual areas including IT and regions of the STS will be made in order to characterize the tuning of single neurons to the shape dynamics of specific image sequences. By combining modeling and physiology, this work will search for a computational explanation for how the higher areas of the visual cortex recognize objects and actions over time and how they can learn. This integrative effort, which is focused on processing of dynamic perceptual information, can have a significant and direct impact on current theories of autism, dyslexia, and effects of stroke, in addition to directly guiding modeling and engineering efforts in computer vision. The proposed research is tightly coupled to education and teaching, and resources used in the research, including databases of videos, visual stimuli, the modeling software and the experimental data will be made available to the broad scientific community. Information on the project and its progress will be available at http://cbcl.mit.edu/projects/NSF-CRCNS/index.html