Collaborative Proposal: Object and Action Recognition in Time Sequences of Images: Computational Neuroscience and Neurophysiology

协作提案：图像时间序列中的对象和动作识别：计算神经科学和神经生理学

• 批准号: 0827483
• 负责人: Tomaso Poggio
• 金额: $ 47.5万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-09-01 至 2012-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0827483&HistoricalAwards=false
• 关键词: Collaborative; Proposal; Object; Action; Recognition

Normal vision is not static: time is a key dimension of the natural world we see. The eventual understanding of biological vision requires understanding the neural mechanisms used to recognize objects and actions over time. Thus the focus of the proposed research is to study how the primate visual system recognizes objects and actions in time sequences of images. A meta-goal of this project is to exploit the synergies between computational approaches and physiological experiments to lead to a better understanding of brain function and at the same time to develop better computer vision algorithms. Object recognition in time sequences of images presents a significant challenge for recognition systems, because it requires both selectivity to shape and invariance to changes of appearance in time.. This project will extend an existing computational model of the ventral stream by adding temporal dynamics in its model neurons and the ability to process video sequences. It will also expand a working model of the dorsal stream to understand the relative roles that it and the ventral stream play in dynamic visual recognition. At the same time, recordings from single units, and multiple single units, from high level visual areas including IT and regions of the STS will be made in order to characterize the tuning of single neurons to the shape dynamics of specific image sequences. By combining modeling and physiology, this work will search for a computational explanation for how the higher areas of the visual cortex recognize objects and actions over time and how they can learn.This integrative effort, which is focused on processing of dynamic perceptual information, can have a significant and direct impact on current theories of autism, dyslexia, and effects of stroke, in addition to directly guiding modeling and engineering efforts in computer vision. The proposed research is tightly coupled to education and teaching, and resources used in the research, including databases of videos, visual stimuli, the modeling software and the experimental data will be made available to the broad scientific community. Information on the project and its progress will be available at http://cbcl.mit.edu/projects/NSF-CRCNS/index.html

正常的视觉不是静态的：时间是我们所看到的自然世界的一个关键维度。生物视觉的最终理解需要理解用于随着时间的推移识别物体和动作的神经机制。因此，建议的研究重点是研究灵长类视觉系统如何识别图像的时间序列中的对象和动作。 该项目的一个元目标是利用计算方法和生理实验之间的协同作用，以更好地了解大脑功能，同时开发更好的计算机视觉算法。图像时间序列中的对象识别对识别系统提出了重大挑战，因为它需要对形状的选择性和对时间上的外观变化的不变性。这个项目将通过在其模型神经元中添加时间动态和处理视频序列的能力来扩展腹侧流的现有计算模型。它还将扩展背侧流的工作模型，以了解它和腹侧流在动态视觉识别中的相对作用。与此同时，记录来自单个单元和多个单个单元的高水平视觉区域，包括IT和STS区域，以表征单个神经元对特定图像序列的形状动力学的调节。 通过结合建模和生理学，这项工作将寻找视觉皮质高级区域如何随着时间的推移识别物体和动作以及它们如何学习的计算解释。这种综合努力专注于动态感知信息的处理，可以对自闭症、诵读困难和中风影响的当前理论产生重大而直接的影响，除了直接指导计算机视觉中的建模和工程工作之外。 拟议的研究与教育和教学紧密结合，研究中使用的资源，包括视频数据库，视觉刺激，建模软件和实验数据，将提供给广大的科学界。有关该项目及其进展情况的信息将在http://cbcl.mit.edu/projects/NSF-CRCNS/index.html上提供