Collaborative Research: NCS-FO: Learning Efficient Visual Representations From Realistic Environments Across Time Scales

计算机视觉算法检查图像，并理解这些图像所描绘的内容。当前的计算机视觉算法能够在许多图像解释任务中以典型中学生的水平解释图像。计算机视觉的最新进展带来了快速的技术进步，这些进步仍在展开，但不仅影响到技术行业，而且影响到教育、国家安全和医疗保健。然而，这些新的算法到目前为止还知之甚少，并且没有描述像典型的中学生这样的自然学习者是如何学习理解视觉世界的。这项提议将认知心理学家、神经科学家和计算机科学家组成的团队聚集在一起，开发一种受人们学习方式启发的新型计算机视觉算法。这一建议的关键见解是，与许多领先的计算机视觉技术不同，人类学习者广泛利用视觉经验的时间结构来提取结构。在现实世界中，人类视网膜上的图像几乎从来都不是静态的。眼睛位置的变化以及头部和身体的运动在从数百毫秒到几天和几周的一系列尺度上创造了丰富而复杂的时间结构。该建议a)开发真实世界和沉浸式虚拟现实环境中真实和动态变化的图像的数据库，b)开发用于从时间结构的经验中学习视觉表示的计算模型，以及c)使用fMRI检查支持跨尺度整合时间和空间的表示的大脑结构。这个项目中追求的算法受到了最近在神经科学中关于尺度不变记忆的理论工作的启发。然而，由于这些数据库将被公开使用，其他研究人员将能够开发其他利用时间和空间相关性的算法。综上所述，这些努力旨在催化新一代类似人类的机器学习算法在计算机视觉中的应用。

Computer vision algorithms examine images and make sense of what these images depict. Current computer vision algorithms are able to interpret images at the level of a typical middle school student for many image interpretation tasks. Recent advances in computer vision have led to rapid technological advances which are still unfolding but affect not only the technology industry, but education, national security and health care. However, these new algorithms are as yet poorly understood and do not describe how natural learners such as a typical middle school student learn to understand the visual world. This proposal draws together a team of cognitive psychologists, neuroscientists, and computer scientists to develop a new class of algorithms for computer vision inspired by the way people learn. The key insight of this proposal is that human learners, unlike many leading computer vision techniques, make extensive use of the temporal structure of visual experience to extract structure. In the real world the image on the human retina is almost never static. Changes in eye position and movements of the head and body create a rich and complex temporal structure over a range of scales from hundreds of milliseconds up to days and weeks. This proposal a) develops databases of realistic and dynamically changing images in the real world and in immersive virtual reality environments, b) develops computational models for learning visual representations from temporally structured experiences and, c) examines the brain structures supporting representations integrating time and space across scales using fMRI. The algorithms pursued in this project are inspired by recent theoretical work in the neuroscience of scale-invariant memory. However, because the databases will be made publicly available, other researchers will be able to develop other algorithms that exploit temporal and spatial correlations. Taken together, these efforts are intended to catalyze a new generation of techniques for human-like machine learning algorithms with applications in computer vision.