RI: Medium: Creating Knowledge with All-Novel-Class Computer Vision

计算机视觉方法已经对科学和工业产生了非常高的影响，但这种影响仅限于可以访问非常大量的标记数据的情况（即，在图像中识别对象），这些对象或者已经出版、被收集或者被购买。这项研究将研究在标记数据很少的领域中运行的计算机视觉方法。这个项目建立在人类和动物学习标记图像的自然模型上。一个核心研究目标是一个可以用所有类别数据训练的对象检测过程-每个类别都有少量的例子。另一个核心目标是一个学习过程，可以广泛有效地跨类别共享训练示例，而无需明确链接类别。第三个核心目标是将早期视觉任务的学习（例如，从图像中恢复阴影和照明）与分类和检测任务的学习联系起来，这样两个任务都可以用很少的标记数据来学习。这项研究的成功完成将通过将早期视觉和分类直接联系起来，统一计算机视觉的明显不同的领域，并将创造新的方法来改善困难情况下的分类性能。此外，这项研究的成功完成将解锁许多需要所有类别方法的实际应用。全新类问题发生在有少量的例子，每个例子来自大量的类，没有类有很多例子。该项目通过在培训过程中共享各种信息来解决全小说类问题。具体而言，将研究三种共享原则。第一个是细胞一致性原则，它使用由特征生成驱动的类边界的几何和概率分析来产生分类的改进，通过要求特征空间中的细胞与相似的类相关联。其次，标签一致性原则使用概率推理来估算未标记示例的标签和置信度权重。标签一致性可以广泛应用，从单个示例的类别标签到标识类的超类标签，在这些类上共享将是有帮助的。最后，一个物理一致性原则要求从图像的推论是符合简单的物理定律;这一原则允许研究人员归咎于早期视力数据的注释丢失，并通过属性理论链接到早期视力的高级类。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Computer vision methods have had very high impact on science and industry, but this impact has been confined to cases where there is access to very large quantities of labelled data (i.e., objects are identified in the image), which have either been published, collected or purchased. This research will study computer vision methods that operate in areas where there are very little labelled data. This project builds on the natural model of the way humans and animals learn to label images. One core research goal is an object detection procedure that can be trained with all category data — there will be a small number of examples each from a large number of categories. Another core goal is a learning procedure that can share training examples across categories widely and effectively without explicit linking of the categories. A third core goal is linking learning of early vision tasks — for example, recovering shading and lighting from an image — to learning of classification and detection tasks, so that both tasks can be learned with very little labelled data. Successful completion of this research will unify apparently disparate areas of computer vision, by linking early vision and categorization directly, and will create novel methods for improving categorization performance in difficult circumstances. Furthermore, successful completion of this research will unlock many real-world applications that need all category methods. The all-novel-class problem occurs where there are a small number of examples each from a large number of classes and no class has many examples. This project addresses the all-novel-class issue by sharing of various kinds of information during training. Specifically, three kinds of sharing principles will be studied. The first is a cell consistency principle that uses a geometric and probabilistic analysis of class boundaries driven by feature generation to produce improvements in classification, by requiring that the cells in feature space associated with the similar classes. Second, a label consistency principle uses probabilistic reasoning to impute labels and confidence weights for unlabeled examples. Label consistency can be applied extensively, from category labels for individual examples to superclass labels that identify classes over which sharing will be helpful. Finally, a physical consistency principle requires that inferences from images are consistent with simple physics laws; this principle allows researchers to impute missing annotations for early vision data and links high level classes to early vision through an attribute theory.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.