Collaborative Research: NCS-FO: Learning Efficient Visual Representations From Realistic Environments Across Time Scales

计算机视觉算法检查图像并理解这些图像所描述的内容。目前的计算机视觉算法能够在典型的中学生水平上解释图像，用于许多图像解释任务。 计算机视觉的最新进展带来了快速的技术进步，这些技术进步仍在不断发展，但不仅影响着技术行业，还影响着教育、国家安全和医疗保健。然而，这些新的算法还没有得到很好的理解，也没有描述自然学习者（如典型的中学生）如何学习理解视觉世界。 该提案汇集了认知心理学家，神经科学家和计算机科学家的团队，以开发一类受人们学习方式启发的新的计算机视觉算法。 这项建议的关键见解是，与许多领先的计算机视觉技术不同，人类学习者广泛使用视觉体验的时间结构来提取结构。 在真实的世界中，人类视网膜上的图像几乎从来都不是静止的。 眼睛位置的变化以及头部和身体的运动在从数百毫秒到数天和数周的范围内创造了丰富而复杂的时间结构。该提案a）开发真实的世界和沉浸式虚拟现实环境中的逼真和动态变化的图像的数据库，B）开发用于从时间结构化体验学习视觉表征的计算模型，以及c）使用fMRI检查支持跨尺度整合时间和空间的表征的大脑结构。在这个项目中追求的算法的灵感来自最近的理论工作，在神经科学的尺度不变的记忆。 然而，由于这些数据库将公开提供，其他研究人员将能够开发其他利用时间和空间相关性的算法。 总而言之，这些努力旨在促进新一代类人机器学习算法技术在计算机视觉中的应用。

Computer vision algorithms examine images and make sense of what these images depict. Current computer vision algorithms are able to interpret images at the level of a typical middle school student for many image interpretation tasks. Recent advances in computer vision have led to rapid technological advances which are still unfolding but affect not only the technology industry, but education, national security and health care. However, these new algorithms are as yet poorly understood and do not describe how natural learners such as a typical middle school student learn to understand the visual world. This proposal draws together a team of cognitive psychologists, neuroscientists, and computer scientists to develop a new class of algorithms for computer vision inspired by the way people learn. The key insight of this proposal is that human learners, unlike many leading computer vision techniques, make extensive use of the temporal structure of visual experience to extract structure. In the real world the image on the human retina is almost never static. Changes in eye position and movements of the head and body create a rich and complex temporal structure over a range of scales from hundreds of milliseconds up to days and weeks. This proposal a) develops databases of realistic and dynamically changing images in the real world and in immersive virtual reality environments, b) develops computational models for learning visual representations from temporally structured experiences and, c) examines the brain structures supporting representations integrating time and space across scales using fMRI. The algorithms pursued in this project are inspired by recent theoretical work in the neuroscience of scale-invariant memory. However, because the databases will be made publicly available, other researchers will be able to develop other algorithms that exploit temporal and spatial correlations. Taken together, these efforts are intended to catalyze a new generation of techniques for human-like machine learning algorithms with applications in computer vision.