CRII: RI: Multi-Source Domain Generalization Approaches to Visual Attribute Detection

CRII：RI：视觉属性检测的多源域泛化方法

• 批准号: 1835539
• 负责人: Boqing Gong
• 金额: $ 4.08万
• 依托单位: International Computer Science Institute
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-12-23 至 2020-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1835539&HistoricalAwards=false
• 关键词: CRII; RI; Multi; Source; Domain

This project investigates how to accurately and robustly detect attributes from images (videos, and 3D data), with the goal of developing and publicly providing effective attribute detection tools. Visual attributes refer to human-namable and machine-detectable inherent characteristics of visual content from objects, scenes, and activities (e.g., four-legged, outdoor, and crowded). They possess versatile properties and application potentials by offering a natural human-computer interaction channel for involving humans in the loop of machine vision algorithms, serving as basic building blocks for one to compose categories and describe instances, and bringing rich prior knowledge and regularization to statistical learning models, to name a few. The project advances the long-standing pursuit of utilizing attributes for a wide variety of visual recognition and search tasks. The project also actively engages graduate and undergraduate students, and outreaches local high-school students. The research results from this project can impact several related communities such as NLP, speech, and robotics, etc.. This research explicitly tackles the need that attribute detectors should generalize well across different categories, including those previously unseen ones. The research team approaches the problem based on multi-source domain generalization by taking each category as a domain. In particular, this project develops new feature extraction tools tailored to account for the middle-level attributes, as opposed to the traditional features primarily designed and tested for high-level visual recognition. The project consists of three major thrusts hinging on the key motivation of the analogy between attribute detection and domain generalization. It begins by learning a fine-grained "shallow" feature mapping (Thrust I) to distill attribute-discriminative signals that are category-invariant, and then investigates "deeper" into the feature extraction frameworks - Fisher vectors (Thrust II) and convolutional neural networks (Thrust III)-to revise them for the purpose of attribute detection.

本项目研究如何从图像（视频和3D数据）中准确和鲁棒地检测属性，目标是开发和公开提供有效的属性检测工具。视觉属性是指来自对象、场景和活动的视觉内容的人类可命名和机器可检测的固有特征（例如，四条腿，户外，拥挤）。它们具有多功能的特性和应用潜力，提供了一个自然的人机交互通道，使人类参与机器视觉算法的循环，作为组成类别和描述实例的基本构建块，并为统计学习模型带来丰富的先验知识和正则化，仅举几例。该项目推进了长期以来对利用属性进行各种视觉识别和搜索任务的追求。该项目还积极吸引研究生和本科生，并扩大到当地高中生。该项目的研究成果可以影响几个相关的社区，如NLP，语音和机器人等。 这项研究明确地解决了属性检测器应该在不同类别中推广的需求，包括那些以前看不见的类别。研究团队通过将每个类别作为一个域，基于多源域泛化来解决问题。特别是，该项目开发了新的特征提取工具，专门针对中级属性，而不是主要为高级视觉识别设计和测试的传统特征。该项目由三个主要的推力铰链属性检测和领域泛化之间的类比的关键动机。它首先学习细粒度的“浅”特征映射（Thrust I）来提取类别不变的属性判别信号，然后“更深入”地研究特征提取框架- Fisher向量（Thrust II）和卷积神经网络（Thrust III）-以修改它们用于属性检测。