RI: Medium: Creating Knowledge with All-Novel-Class Computer Vision

RI：媒介：利用新颖的计算机视觉创造知识

基本信息

批准号：
2106825
负责人：
David Forsyth
金额：
$ 120万
依托单位：
University of Illinois at Urbana-Champaign
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-09-01 至 2025-08-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106825&HistoricalAwards=false
关键词：
RI Medium Creating Knowledge Novel

项目摘要

Computer vision methods have had very high impact on science and industry, but this impact has been confined to cases where there is access to very large quantities of labelled data (i.e., objects are identified in the image), which have either been published, collected or purchased. This research will study computer vision methods that operate in areas where there are very little labelled data. This project builds on the natural model of the way humans and animals learn to label images. One core research goal is an object detection procedure that can be trained with all category data — there will be a small number of examples each from a large number of categories. Another core goal is a learning procedure that can share training examples across categories widely and effectively without explicit linking of the categories. A third core goal is linking learning of early vision tasks — for example, recovering shading and lighting from an image — to learning of classification and detection tasks, so that both tasks can be learned with very little labelled data. Successful completion of this research will unify apparently disparate areas of computer vision, by linking early vision and categorization directly, and will create novel methods for improving categorization performance in difficult circumstances. Furthermore, successful completion of this research will unlock many real-world applications that need all category methods. The all-novel-class problem occurs where there are a small number of examples each from a large number of classes and no class has many examples. This project addresses the all-novel-class issue by sharing of various kinds of information during training. Specifically, three kinds of sharing principles will be studied. The first is a cell consistency principle that uses a geometric and probabilistic analysis of class boundaries driven by feature generation to produce improvements in classification, by requiring that the cells in feature space associated with the similar classes. Second, a label consistency principle uses probabilistic reasoning to impute labels and confidence weights for unlabeled examples. Label consistency can be applied extensively, from category labels for individual examples to superclass labels that identify classes over which sharing will be helpful. Finally, a physical consistency principle requires that inferences from images are consistent with simple physics laws; this principle allows researchers to impute missing annotations for early vision data and links high level classes to early vision through an attribute theory.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机视觉方法对科学和工业产生了很大的影响，但是这种影响仅限于可以访问大量标记的数据（即在图像中确定的对象）的情况，这些数据已发布，收集或购买。这项研究将研究在很少有标记数据的领域运行的计算机视觉方法。该项目以人类和动物学习标记图像的方式的自然模型为基础。一个核心研究目标是一个可以使用所有类别数据训练的对象检测过程 - 每个类别中将有少数示例。另一个核心目标是一个学习过程，可以在不明确链接类别的情况下广泛有效地分享培训示例。第三个核心目标是链接对早期视觉任务的学习，例如，从图像中恢复阴影和照明 - 与分类和检测任务的学习，以便可以通过很少的标记数据来学习这两个任务。这项研究的成功完成将通过直接联系早期视觉和类别来统一计算机视觉的明显不同领域，并将创建新的方法来改善困难情况下的类别性能。此外，这项研究的成功完成将解锁许多需要所有类别方法的现实世界应用程序。全都级问题发生在大量课程中每个示例中都有少量示例，而没有类别的示例有很多示例。该项目通过在培训期间共享各种信息来解决全都级问题。具体来说，将研究三种共享原则。第一个是单元一致性原理，该原理使用对特征生成驱动的类边界的几何和有问题的分析来通过要求与类似类相关的特征空间中的细胞来产生分类的改进。其次，标签一致性原理使用概率推理将标签和置信权重定为未标记的示例。可以广泛应用标签一致性，从单个示例的类别标签到识别共享的类有帮助的类的超类标签。最后，身体一致性原则要求图像的推论与简单的物理定律一致。该原则使研究人员能够通过属性理论将缺少的注释归为早期视觉数据，并将高级类别与早期愿景联系起来。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子和更广泛的影响评估标准来通过评估来表示赞成支持。