CIF: Small: An Information Theoretic Framework for Minimizing Supervision in Image/Video Analysis

最近在图像和视频分析方面的成功很大程度上是在监督学习领域。监督学习方法假设有大量的人工标注/标记的训练数据可用，这限制了现有方法对复杂和不可见环境的适用性。这激发了人们对开发半监督甚至无监督的图像和视频分析方法的日益浓厚的兴趣，即，具有有限甚至没有人工注释数据的方法。这些方法侧重于如何在有限的标记数据下学习视觉分析模型；然而，对标记什么的问题解决得很少。如果一个人能够确定要标记的最佳子集，那么学习过程很可能比随机选择由人类标记的代表更有效。这个项目将集中于如何选择这些样本进行标记的严格数学方法。该项目将调查信息论方法，并了解视觉数据的内在结构，这将有助于识别需要注释的样本。信息论中一个广为人知的概念是典型集合。典型集合的概念基于这样一个直观的概念，即数据集的某些部分比其他部分承载更多的信息。该项目将调查信息论中的这一典型性概念的适用性，以选择信息量最大的样本的最小集合，这些样本将被手动标记，以便在这个子集上设计的视觉分析模型(例如，分类器)可以在整个数据集上最大化(感兴趣的任务)的性能。视觉数据中的内在结构将在这个信息论框架内结合起来，以获得要标记的最优代表集。该方法将在各种学习范式的背景下进行分析，包括主动学习、迁移学习和弱监督学习。该项目将包括对代表性选择过程的理论分析、算法设计，以及对图像分割、活动识别和目标重新识别应用程序的实验评估。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The recent successes in image and video analysis have been largely in the domain of supervised learning. Supervised learning methods assume the availability of extensive amounts of manually annotated/labeled training data, which limits the applicability of existing methods to complex and unseen environments. This has motivated growing interest in developing semi-supervised, and even unsupervised, methods for image and video analysis, i.e., methods that have limited or even no manually annotated data. These methods focus on how to learn visual analysis models with limited labeled data; however, the problem of what to label is far less addressed. If one can identify the optimal subset to label, it is likely that the learning process will be more efficient than randomly choosing representatives that are labeled by a human. This project will focus on mathematically rigorous approaches on how to choose these samples to label.The project will investigate information theoretic approaches, coupled with an understanding of the inherent structure in visual data, that would help identify samples that require annotation. A well-known idea in information theory is typical sets. The concept of a typical set is based on the intuitive notion that some portions of a dataset carry more information than others. The project will investigate the applicability of this notion of typicality from information theory to select a minimal set of most informative samples, which will be manually labeled, such that visual analysis models (e.g., classifiers) designed on this subset can maximize performance (of the task of interest) on the entire dataset. The inherent structure in visual data will be combined within this information theoretic framework to obtain the optimal set of representatives to label. The approach will be analyzed in the context of various learning paradigms, including active learning, transfer learning, and weakly supervised learning. The project will involve theoretical analysis of the representative selection process, algorithm design, and experimental evaluation on applications in image segmentation, activity recognition, and target re-identification.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.