RI: Medium: Creating Knowledge with All-Novel-Class Computer Vision

点击翻译按钮获取中文摘要

Computer vision methods have had very high impact on science and industry, but this impact has been confined to cases where there is access to very large quantities of labelled data (i.e., objects are identified in the image), which have either been published, collected or purchased. This research will study computer vision methods that operate in areas where there are very little labelled data. This project builds on the natural model of the way humans and animals learn to label images. One core research goal is an object detection procedure that can be trained with all category data — there will be a small number of examples each from a large number of categories. Another core goal is a learning procedure that can share training examples across categories widely and effectively without explicit linking of the categories. A third core goal is linking learning of early vision tasks — for example, recovering shading and lighting from an image — to learning of classification and detection tasks, so that both tasks can be learned with very little labelled data. Successful completion of this research will unify apparently disparate areas of computer vision, by linking early vision and categorization directly, and will create novel methods for improving categorization performance in difficult circumstances. Furthermore, successful completion of this research will unlock many real-world applications that need all category methods. The all-novel-class problem occurs where there are a small number of examples each from a large number of classes and no class has many examples. This project addresses the all-novel-class issue by sharing of various kinds of information during training. Specifically, three kinds of sharing principles will be studied. The first is a cell consistency principle that uses a geometric and probabilistic analysis of class boundaries driven by feature generation to produce improvements in classification, by requiring that the cells in feature space associated with the similar classes. Second, a label consistency principle uses probabilistic reasoning to impute labels and confidence weights for unlabeled examples. Label consistency can be applied extensively, from category labels for individual examples to superclass labels that identify classes over which sharing will be helpful. Finally, a physical consistency principle requires that inferences from images are consistent with simple physics laws; this principle allows researchers to impute missing annotations for early vision data and links high level classes to early vision through an attribute theory.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.