RI: Small: Collaborative Research: A Scalable Architecture for Image Interpretation

对视觉图像意义的无缝理解是人类认知的一个关键属性，远远超出了当前计算机视觉程序的能力。 该项目的目的是建立一个计算系统，通过将较低层次的视觉感知与较高层次的概念相结合，来捕捉人类视觉的动态和交互方面。 如果成功的话，这个系统将能够以一种与场景的复杂性很好地匹配的方式来解释视觉场景。当前的计算机视觉系统通常依赖于相对低级的视觉信息（例如，颜色、纹理、形状）来对对象进行分类或确定场景的总体类别。 这种分类通常以“自下而上”的方式进行，其中视觉系统从场景的所有部分提取较低级别的特征，并随后分析提取的特征以确定场景的哪些部分包含感兴趣的对象以及应该如何对这些对象进行分类。 这样的系统缺乏扩展到大量的视觉类别和识别更复杂的视觉概念，涉及对象类别之间的空间和抽象关系的能力。 人类的视觉感知是一个具有反馈的时间过程，其中较低级别的视觉特征用于激活较高级别的概念（或知识）。 这些积极的概念，反过来，指导的看法和注意力给予较低层次的视觉特征。 此外，激活的概念可以将激活传播到语义相关的概念（例如，“wheels”可以激活“car”或“bicycle”;“bicycle”可以激活“road”或“rider”）。 通过这种方式，在较低和较高的视觉水平之间存在持续的相互作用，这使得观看者能够专注于并连接复杂场景的重要方面，以便感知其含义，而不必同等关注场景的每个细节。 这里提出的系统将模拟人类视觉感知的这些方面。 该系统名为Petacat，将整合并建立在两个现有项目的基础上：最初由Riesenhuber和Poggio开发的物体识别HMAX模型，以及由Hofstadter和Mitchell开发的高级感知和类比制作的模仿模型。 HMAX通过前馈网络模拟哺乳动物视觉皮层的“什么”通路，该网络从图像中提取越来越复杂的纹理和形状特征。(HMAX已经被洛斯阿拉莫斯的合成视觉小组重新实现为"Petascale人工神经网络"或PANN，以允许在大量神经元上进行高性能计算。 模仿实现了高层次概念和低层次感知之间的交互过程，并已被用于在几个非视觉领域建模注意力集中，概念滑动和类比。 该项目将结合HMAX/PANN的特征提取能力和Copycat的高级交互感知能力来构建Petacat架构。Petacat的图像解读能力将在相关语义视觉识别任务系列中进行评估（例如，以灵活的、类似人类的方式识别"遛狗"的实例）。 该项目的评估部分将涉及创建图像数据库，用于对语义图像理解系统进行基准测试。 Petacat源代码和基准数据库将通过网络提供。

Seamless understanding of the meaning of visual images is a key property of human cognition that is far beyond the abilities of current computer vision programs. The purpose of this project is to build a computational system that captures the dynamical and interactive aspects of human vision by integrating higher-level concepts with lower-level visual perception. If successful, this system will be able to interpret visual scenes in a way that scales well with the complexity of the scene. Current computer vision systems typically rely on relatively low-level visual information (e.g., color, texture, shape) to classify objects or determine the overall category of a scene. Such categorization is typically done in a "bottom-up" fashion, in which the vision system extracts lower-level features from all parts of the scene, and subsequently analyzes the extracted features to determine which parts of the scene contain objects of interest and how those objects should be categorized. Such systems lack the abilities to scale to large numbers of visual categories and to identify more complex visual concepts that involve spatial and abstract relationships among object categories. Visual perception by humans is known to be a temporal process with feedback, in which lower-level visual features serve to activate higher-level concepts (or knowledge). These active concepts, in turn, guide the perception of and attention given to lower-level visual features. Moreover, activated concepts can spread activation to semantically related concepts (e.g., "wheels" might activate "car" or "bicycle"; "bicycle" might activate "road" or "rider"). In this way there is a continual interaction between the lower and higher levels of vision, which allows the viewer to focus on and connect important aspects of a complex scene in order to perceive its meaning, without having to pay equal attention to every detail of the scene. The system proposed here will model these aspects of human visual perception. The proposed system, called Petacat, will integrate and build on two existing projects: the HMAX model of object recognition originally developed by Riesenhuber and Poggio, and the Copycat model of high-level perception and analogy-making, developed by Hofstadter and Mitchell. HMAX models the "what" pathway of mammalian visual cortex via a feed-forward network that extracts increasingly complex textural and shape features from an image. (HMAX has been reimplemented, as the "Petascale Artificial Neural Network" or PANN, by the Synthetic Vision Group at Los Alamos to allow for high-performance computing on large numbers of neurons.) Copycat implements a process of interaction between high-level concepts and lower-level perception, and has been used to model focus of attention, conceptual slippage, and analogy-making in several non-visual domains. This project will marry the feature extraction abilities of HMAX/PANN with the higher-level interactive perceptual abilities of Copycat to build the Petacat architecture. The image interpretation abilities of Petacat will be evaluated on families of related semantic visual recognition tasks (e.g., recognizing, in a flexible, human-like way, instances of "walking a dog"). The evaluation part of the project will involve the creation of image databases for benchmarking semantic image-understanding systems. The Petacat source code and benchmarking databases will be made publically available via the web.