CompCog: Human Scene Processing Characterized by Computationally-derived Scene Primitives

我们的大脑是如何将进入我们眼睛的光线转化为我们对周围世界的体验的？关键的是，这种体验似乎包含了一种视觉“词汇”，让我们能够根据先前的知识理解新的场景。研究人员探索了这种视觉语言的本质，探索了在用于场景感知的大脑机制中实现的特定计算。这项工作将来自最先进的计算机视觉系统的数据与人类神经成像相结合，既可以预测观看复杂的真实世界场景时的大脑反应，也可以分析和理解嵌入真实世界图像中的隐藏结构。这一努力对于建立我们如何能够观看的理论和改进机器视觉系统是至关重要的。更广泛地说，受生物启发的视觉模型对于智能技术在导航系统、辅助设备、安全验证和视觉信息检索中的有效部署至关重要。本研究采用的人工视觉系统是高度数据驱动的，因为它通过不断地在万维网上观看真实世界的图像来学习视觉世界。该模型名为“Neil”(Never Ending Image Learner)，http://www.neil-kb.com/)，利用尖端的大数据方法从成千上万张图像中提取场景部分和关系的词汇表。然后将使用功能磁共振成像(FMRI)和脑磁图(MEG)神经成像来测试这些词汇与人类视觉的相关性。这个假设是，关于场景的先验知识的应用是通过特定部分和关系之间的习得联系来表达自己，形成了场景感知的词汇。此外，不同类型的关联可以在负责场景感知的功能大脑网络的不同组件中实例化。总体而言，这项研究将建立在最近一个非常成功的人工视觉系统的基础上，以提供一个更具体的关于人类场景感知的部分和关系的理论。同时，这项研究将提供有关通过计算得出的场景部分和关系的人类功能相关性的信息，从而有助于改进和改进人工视觉系统。

How do our brains take the light entering our eyes and turn it into our experience of the world around us? Critically, this experience seems to involve a visual "vocabulary" that allows us to understand new scenes based on our prior knowledge. The investigators explore the nature of this visual language, exploring the specific computations that are realized in the brain mechanisms used for scene perception. The work combines data from state-of-the-art computer vision systems with human neuroimaging to both predict brain responses when viewing complex, real-world scenes, and to analyze and understand the hidden structure embedded in real-world images. This effort is essential for building a theory of how we are able to see and for improving machine vision systems. More broadly, biologically-inspired models of vision are essential for the effective deployment of intelligent technology in navigation systems, assistive devices, security verification, and visual information retrieval.The artificial vision system adopted in this research is highly data-driven in that it is learning about the visual world by continuously "looking at" real-world images on the World Wide Web. The model, known as "NEIL" (Never Ending Image Learner, http://www.neil-kb.com/), leverages cutting-edge big-data methods to extract a vocabulary of scene parts and relationships from hundreds of thousands of images. The relevance of this vocabulary to human vision will then be tested using both functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) neuroimaging. The hypothesis is that the application of prior knowledge about scenes expresses itself through learned associations between the specific parts and relations forming the vocabulary for scene perception. Moreover, different kinds of associations may be instantiated within distinct components of the functional brain network responsible for scene perception. Overall, this research will build on a recent, highly-successful artificial vision system in order to provide a more well-specified theory of the parts and relations underlying human scene perception. At the same time, the research will provide information about the human functional relevance of computationally-derived scene parts and relations, thereby helping to refine and improve artificial vision systems.