RI: Small: Visual Situation Recognition: An Integration of Deep Networks and Analogy-Making

这个项目研究了一种新的方法来构建可以识别视觉情况的计算机系统。 虽然计算机视觉的许多努力都集中在识别图像中的孤立对象上，但人们实际上所做的是识别连贯的情况-对象的集合及其相互关系，放在一起，对应于一个已知的概念，例如“孩子的生日派对”，或“一个人在海滩遛狗”，或“两个人即将打架”，或“一个盲人过马路”。“人类的情况识别表面上看起来似乎毫不费力，但它依赖于人类感知物体的能力之间复杂的动态相互作用，物体之间的关系系统，以及与存储的知识和记忆的类比。 目前还没有计算机视觉系统能够捕捉到人类的这些能力。 使计算机能够灵活地识别视觉情况将在医疗诊断、科学图像解释、增强人机交互和个人信息组织等领域创造大量重要的应用。本项目探索的方法整合了两种先前研究的方法：用于低层次视觉的脑启发神经网络和概念和类比的认知层次模型。 在这种综合架构中，通过与存储的概念结构进行类比来识别情况将是一个动态过程，在这个过程中，自下而上（感知）和自上而下（概念）的影响随着感知的展开而相互影响。 如果成功的话，这个系统将能够以一种与场景的复杂性和被识别的抽象概念很好地匹配的方式来识别视觉场景。 作为该项目的一部分，将收集一些反映不同抽象视觉情况的基准图像数据集，以评估识别系统。 此外，PI将使用收集的数据集设计和运行一个关于视觉情况自动识别的公开竞赛。 这项比赛将刺激对这一主题的研究，并帮助在这一领域工作的研究人员评估各种方法的成功，并衡量抽象视觉识别的最新技术水平。 在这个项目中开发的所有源代码和基准数据库将通过网络公开。

This project investigates a novel approach to building computer systems that can recognize visual situations. While much effort in computer vision has focused on identifying isolated objects in images, what people actually do is recognize coherent situations--collections of objects and their interrelations that, taken together, correspond to a known concept, such as "a child's birthday party," or "a man walking a dog on the beach," or "two people about to fight," or "a blind person crossing the street." Situation recognition by humans may appear on the surface to be effortless, but it relies on a complex dynamic interplay among human abilities to perceive objects, systems of relationships among objects, and analogies with stored knowledge and memories. No computer vision system yet comes close to capturing these human abilities. Enabling computers to flexibly recognize visual situations would create a flood of important applications in fields as diverse as medical diagnosis, interpretation of scientific imagery, enhanced human-computer interaction, and personal information organization.The approach explored in this project integrates two previously studied approaches: brain-inspired neural networks for lower-level vision and cognitive-level models of concepts and analogy-making. In this integrated architecture, recognizing situations--via analogies with stored conceptual structures--will be a dynamic process in which bottom-up (perceptual) and top-down (conceptual) influences affect one another as perception unfolds. If successful, this system will be able to recognize visual situations in a way that scales well with the complexity of the scene and the abstract concept being recognized. As part of this project, a number of benchmark image datasets--reflecting different abstract visual situations--will be collected to evaluate the recognition system. In addition, the PI will design and run a public competition on automated recognition of visual situations, using the collected datasets. This competition will spur research on this topic, and help researchers working in this area evaluate the success of various methods and gauge the current state of the art on abstract visual recognition. All source code and benchmarking databases developed in this project will be made publicly available via the web.