RI: Small: Advancing Visual Recognition with Feature Visualizations

RI：小型：通过特征可视化推进视觉识别

• 批准号: 1524817
• 负责人: Antonio Torralba
• 金额: $ 46万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1524817&HistoricalAwards=false
• 关键词: RI; Small; Advancing; Visual; Recognition

The goal of this work is to develop a set of tools to visualize the information extracted by computer vision systems so that it is easier for researchers, and users, to understand their behavior. With the success of new computational architectures for visual processing, such as deep neural networks with many processing layers (e.g., convolutional neural networks) and access to large databases with millions of annotated images (e.g., ImageNet, Places), the state of the art in computer vision is advancing rapidly, and becoming integrated into many commercial products. But these advances come with the price that systems are becoming more complex, and it becomes harder for researchers and users to diagnose and understand the representations built by these systems. The goal of this work is to develop new techniques for visualizing what the algorithms are doing in order to elucidate their behavior. The work will focus on developing algorithms for generic feature inversion. Most features perform complex non-linear operations over the image and it is not always possible to obtain analytic expressions to invert those computations. The goal of the proposal is to introduce new techniques that will allow inverting descriptors without constraining the descriptors. The second challenge will consists in understanding the inversion properties in order to allow comparison among different descriptions. If the inversion contains approximations, comparisons among descriptors might not be possible. Therefore it will be important to understand the convergence properties of the inversion algorithms. Another issue arises from the compressive nature of most descriptors. In general, some part of the input image information will be lost when encoded by an image descriptor. Therefore, the inversion will have to be a one-to-many function. Understanding the space of equivalent images under a particular descriptor will provide insights about what will be the likely errors made by a recognition system using them. This proposal will perform a variety of experiments with the feature visualizations, such as examining invariances in both engineered features and learned features from deep learning, visualizing learned models and decision boundaries, and diagnosing false alarms and missed detections.

这项工作的目标是开发一套工具来可视化计算机视觉系统提取的信息，以便研究人员和用户更容易理解他们的行为。随着视觉处理的新计算架构的成功，例如具有许多处理层的深度神经网络（例如，卷积神经网络）和访问具有数百万注释图像的大型数据库（例如，ImageNet, Places），计算机视觉的最新技术正在迅速发展，并已集成到许多商业产品中。但这些进步的代价是系统变得越来越复杂，研究人员和用户越来越难以诊断和理解这些系统构建的表征。这项工作的目标是开发新的技术来可视化算法正在做什么，以阐明它们的行为。这项工作将侧重于开发通用特征反演算法。大多数特征在图像上执行复杂的非线性操作，并且并不总是可能获得解析表达式来反转这些计算。该提案的目标是引入新的技术，允许在不限制描述符的情况下反转描述符。第二个挑战将包括理解反转属性，以便在不同的描述之间进行比较。如果反演包含近似值，描述符之间的比较可能是不可能的。因此，了解反演算法的收敛性是很重要的。另一个问题来自大多数描述符的压缩特性。通常，在使用图像描述符进行编码时，会丢失部分输入图像信息。因此，反转必须是一对多函数。了解特定描述符下的等效图像空间将提供关于使用它们的识别系统可能产生的错误的见解。该提案将使用特征可视化执行各种实验，例如检查工程特征和深度学习学习特征的不变性，可视化学习模型和决策边界，以及诊断假警报和遗漏检测。