CompCog: Human Scene Processing Characterized by Computationally-derived Scene Primitives

我们的大脑如何接收进入眼睛的光线，并将其转化为我们对周围世界的体验？至关重要的是，这种体验似乎涉及到一种视觉“词汇”，使我们能够基于先前的知识理解新场景。研究人员探索了这种视觉语言的本质，探索了在用于场景感知的大脑机制中实现的具体计算。这项工作将来自最先进的计算机视觉系统的数据与人类神经成像相结合，既可以预测大脑在观看复杂的现实世界场景时的反应，也可以分析和理解嵌入在现实世界图像中的隐藏结构。这项工作对于建立我们如何能够看到的理论和改进机器视觉系统至关重要。更广泛地说，生物启发的视觉模型对于在导航系统、辅助设备、安全验证和视觉信息检索中有效部署智能技术至关重要。本研究中采用的人工视觉系统是高度数据驱动的，它通过不断“查看”万维网上的真实图像来学习视觉世界。该模型被称为“NEIL”（Never Ending Image Learner, http://www.neil-kb.com/），利用尖端的大数据方法从数十万张图像中提取场景部分和关系的词汇表。然后将使用功能磁共振成像（fMRI）和脑磁图（MEG）神经成像来测试这些词汇与人类视觉的相关性。该假说认为，对场景的先验知识的应用是通过特定部分之间的习得关联和形成场景感知词汇的关系来表达的。此外，不同类型的关联可能在负责场景感知的功能性大脑网络的不同组成部分中实例化。总的来说，这项研究将建立在最近非常成功的人工视觉系统的基础上，以便为人类场景感知的部分和关系提供更详细的理论。同时，该研究将提供有关计算导出的场景部分和关系的人类功能相关性的信息，从而有助于改进和改进人工视觉系统。

How do our brains take the light entering our eyes and turn it into our experience of the world around us? Critically, this experience seems to involve a visual "vocabulary" that allows us to understand new scenes based on our prior knowledge. The investigators explore the nature of this visual language, exploring the specific computations that are realized in the brain mechanisms used for scene perception. The work combines data from state-of-the-art computer vision systems with human neuroimaging to both predict brain responses when viewing complex, real-world scenes, and to analyze and understand the hidden structure embedded in real-world images. This effort is essential for building a theory of how we are able to see and for improving machine vision systems. More broadly, biologically-inspired models of vision are essential for the effective deployment of intelligent technology in navigation systems, assistive devices, security verification, and visual information retrieval.The artificial vision system adopted in this research is highly data-driven in that it is learning about the visual world by continuously "looking at" real-world images on the World Wide Web. The model, known as "NEIL" (Never Ending Image Learner, http://www.neil-kb.com/), leverages cutting-edge big-data methods to extract a vocabulary of scene parts and relationships from hundreds of thousands of images. The relevance of this vocabulary to human vision will then be tested using both functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) neuroimaging. The hypothesis is that the application of prior knowledge about scenes expresses itself through learned associations between the specific parts and relations forming the vocabulary for scene perception. Moreover, different kinds of associations may be instantiated within distinct components of the functional brain network responsible for scene perception. Overall, this research will build on a recent, highly-successful artificial vision system in order to provide a more well-specified theory of the parts and relations underlying human scene perception. At the same time, the research will provide information about the human functional relevance of computationally-derived scene parts and relations, thereby helping to refine and improve artificial vision systems.