Collaborative Research: NCS-FO: Learning Efficient Visual Representations From Realistic Environments Across Time Scales

计算机视觉算法检查图像并理解这些图像所描绘的内容。目前的计算机视觉算法能够在许多图像解释任务中达到典型中学生的水平。计算机视觉的最新进展导致了快速的技术进步，这些进步仍在展开，但不仅影响了技术行业，还影响了教育、国家安全和医疗保健。然而，这些新算法还没有被很好地理解，也没有描述自然学习者（如典型的中学生）如何学习理解视觉世界。这个提议吸引了一个由认知心理学家、神经科学家和计算机科学家组成的团队，他们受人类学习方式的启发，为计算机视觉开发了一类新的算法。该建议的关键观点是，与许多领先的计算机视觉技术不同，人类学习者广泛使用视觉经验的时间结构来提取结构。在现实世界中，人类视网膜上的图像几乎从来不是静态的。眼睛位置的变化以及头部和身体的运动创造了丰富而复杂的时间结构，时间跨度从几百毫秒到几天甚至几周不等。本提案a)在现实世界和沉浸式虚拟现实环境中开发现实和动态变化图像的数据库，b)开发用于从时间结构经验中学习视觉表征的计算模型，c)使用功能磁共振成像检查支持跨尺度时空整合表征的大脑结构。在这个项目中所追求的算法受到最近在尺度不变记忆的神经科学理论工作的启发。然而，由于数据库将是公开的，其他研究人员将能够开发利用时间和空间相关性的其他算法。综上所述，这些努力旨在催化新一代的类人机器学习算法技术，并将其应用于计算机视觉。

Computer vision algorithms examine images and make sense of what these images depict. Current computer vision algorithms are able to interpret images at the level of a typical middle school student for many image interpretation tasks. Recent advances in computer vision have led to rapid technological advances which are still unfolding but affect not only the technology industry, but education, national security and health care. However, these new algorithms are as yet poorly understood and do not describe how natural learners such as a typical middle school student learn to understand the visual world. This proposal draws together a team of cognitive psychologists, neuroscientists, and computer scientists to develop a new class of algorithms for computer vision inspired by the way people learn. The key insight of this proposal is that human learners, unlike many leading computer vision techniques, make extensive use of the temporal structure of visual experience to extract structure. In the real world the image on the human retina is almost never static. Changes in eye position and movements of the head and body create a rich and complex temporal structure over a range of scales from hundreds of milliseconds up to days and weeks. This proposal a) develops databases of realistic and dynamically changing images in the real world and in immersive virtual reality environments, b) develops computational models for learning visual representations from temporally structured experiences and, c) examines the brain structures supporting representations integrating time and space across scales using fMRI. The algorithms pursued in this project are inspired by recent theoretical work in the neuroscience of scale-invariant memory. However, because the databases will be made publicly available, other researchers will be able to develop other algorithms that exploit temporal and spatial correlations. Taken together, these efforts are intended to catalyze a new generation of techniques for human-like machine learning algorithms with applications in computer vision.