RI: Small: Collaborative Research: A Scalable Architecture for Image Interpretation

对视觉图像意义的无缝理解是人类认知的一个关键属性，远远超出了当前计算机视觉程序的能力。这个项目的目的是建立一个计算系统，通过整合高层次的概念和低层次的视觉感知来捕捉人类视觉的动态和交互方面。如果成功，该系统将能够以一种与场景复杂性相适应的方式解释视觉场景。当前的计算机视觉系统通常依赖于相对低级的视觉信息（例如，颜色，纹理，形状）来对物体进行分类或确定场景的整体类别。这种分类通常以“自下而上”的方式完成，其中视觉系统从场景的所有部分提取较低层次的特征，然后分析提取的特征以确定场景的哪些部分包含感兴趣的对象以及如何对这些对象进行分类。这样的系统缺乏扩展到大量视觉类别和识别涉及对象类别之间的空间和抽象关系的更复杂的视觉概念的能力。人类的视觉感知是一个具有反馈的时间过程，在这个过程中，较低层次的视觉特征用于激活较高层次的概念（或知识）。这些活跃的概念反过来又引导对较低层次视觉特征的感知和注意。此外，激活的概念可以将激活传播到语义相关的概念（例如，“车轮”可能激活“汽车”或“自行车”；“自行车”可能激活“道路”或“骑手”）。通过这种方式，在较低和较高的视觉层次之间存在持续的互动，这使得观看者能够专注于并连接复杂场景的重要方面，以便感知其意义，而不必同样关注场景的每个细节。这里提出的系统将模拟人类视觉感知的这些方面。这个被称为Petacat的系统将整合并建立在两个现有项目的基础上：最初由Riesenhuber和Poggio开发的对象识别HMAX模型，以及由Hofstadter和Mitchell开发的高级感知和类比模型Copycat模型。HMAX通过一个前馈网络来模拟哺乳动物视觉皮层的“什么”通路，该网络从图像中提取出越来越复杂的纹理和形状特征。（HMAX已经被洛斯阿拉莫斯的合成视觉小组重新实现，称为“千万亿次人工神经网络”或PANN，以允许在大量神经元上进行高性能计算。）Copycat实现了高级概念和低级感知之间的交互过程，并被用于模拟几个非视觉领域的注意力焦点、概念滑移和类比制作。本项目将HMAX/PANN的特征提取能力与Copycat更高层次的交互感知能力相结合，构建Petacat架构。Petacat的图像解释能力将在相关的语义视觉识别任务中进行评估（例如，以灵活的、类似人类的方式识别“遛狗”的实例）。该项目的评估部分将涉及为语义图像理解系统的基准测试创建图像数据库。Petacat的源代码和基准数据库将通过网络公开。

Seamless understanding of the meaning of visual images is a key property of human cognition that is far beyond the abilities of current computer vision programs. The purpose of this project is to build a computational system that captures the dynamical and interactive aspects of human vision by integrating higher-level concepts with lower-level visual perception. If successful, this system will be able to interpret visual scenes in a way that scales well with the complexity of the scene. Current computer vision systems typically rely on relatively low-level visual information (e.g., color, texture, shape) to classify objects or determine the overall category of a scene. Such categorization is typically done in a "bottom-up" fashion, in which the vision system extracts lower-level features from all parts of the scene, and subsequently analyzes the extracted features to determine which parts of the scene contain objects of interest and how those objects should be categorized. Such systems lack the abilities to scale to large numbers of visual categories and to identify more complex visual concepts that involve spatial and abstract relationships among object categories. Visual perception by humans is known to be a temporal process with feedback, in which lower-level visual features serve to activate higher-level concepts (or knowledge). These active concepts, in turn, guide the perception of and attention given to lower-level visual features. Moreover, activated concepts can spread activation to semantically related concepts (e.g., "wheels" might activate "car" or "bicycle"; "bicycle" might activate "road" or "rider"). In this way there is a continual interaction between the lower and higher levels of vision, which allows the viewer to focus on and connect important aspects of a complex scene in order to perceive its meaning, without having to pay equal attention to every detail of the scene. The system proposed here will model these aspects of human visual perception. The proposed system, called Petacat, will integrate and build on two existing projects: the HMAX model of object recognition originally developed by Riesenhuber and Poggio, and the Copycat model of high-level perception and analogy-making, developed by Hofstadter and Mitchell. HMAX models the "what" pathway of mammalian visual cortex via a feed-forward network that extracts increasingly complex textural and shape features from an image. (HMAX has been reimplemented, as the "Petascale Artificial Neural Network" or PANN, by the Synthetic Vision Group at Los Alamos to allow for high-performance computing on large numbers of neurons.) Copycat implements a process of interaction between high-level concepts and lower-level perception, and has been used to model focus of attention, conceptual slippage, and analogy-making in several non-visual domains. This project will marry the feature extraction abilities of HMAX/PANN with the higher-level interactive perceptual abilities of Copycat to build the Petacat architecture. The image interpretation abilities of Petacat will be evaluated on families of related semantic visual recognition tasks (e.g., recognizing, in a flexible, human-like way, instances of "walking a dog"). The evaluation part of the project will involve the creation of image databases for benchmarking semantic image-understanding systems. The Petacat source code and benchmarking databases will be made publically available via the web.