RI: Medium: Collaborative Research: Semantically Discriminative : Guiding Mid-Level Representations for Visual Object Recognition with External Knowledge

这个项目探讨（半）自动的方法来创建“语义歧视”的中级线索视觉对象分类，通过引入外部知识的对象属性的统计学习程序，学习区分它们。 具体而言，PI研究了四个关键思想：（1）利用对象类别上的分类法来通知特征选择算法，以便它们针对给定的标签预测粒度进行最抽象的描述;（2）利用相同分类法传达的对象间关系来指导上下文学习，以便它捕获比简单的数据驱动的同现更多的内容;（3）探索从自然语言中提取的视觉属性的效用，既作为用于对象分类的偏置模型的辅助学习问题，又作为必须使用非传统人类监督策略梳理的序数属性;（4）挖掘既有区别又有人类可命名的属性，该项目在计算机视觉和机器学习方面都有原创性的贡献，是朝着基于语义的对象分类迈出的重要一步。 虽然主流方法将人类知识简化为样本上的类别标签，但这项工作在学习管道中更深入，更早地利用语义丰富的知识。该方法的结果是在视觉系统，不太容易过拟合附带的视觉模式，和表示，很容易扩展到新的视觉学习任务。 在研究界之外，这项工作通过对研究生和本科生的跨学科培训产生了更广泛的影响，并通过讲习班和夏令营向大学预科教育工作者和学生推广，鼓励年轻学生追求科学和工程。

This project explores (semi-)automatic ways to create "semantically discriminative" mid-level cues for visual object categorization, by introducing external knowledge of object properties into the statistical learning procedures that learn to distinguish them. In particular, the PIs investigate four key ideas: (1) exploiting taxonomies over object categories to inform feature selection algorithms such that they home in on the most abstract description for a given granularity of label predictions; (2) leveraging inter-object relationships conveyed by the same taxonomies to guide context learning, so that it captures more than simple data-driven co-occurrences; (3) exploring the utility of visual attributes drawn from natural language, both as auxiliary learning problems to bias models for object categorization, as well as ordinal properties that must be teased out using non-traditional human supervision strategies; (4) mining attributes that are both distinctive and human-nameable, moving beyond manually constructed semantics.The project entails original contributions in both computer vision and machine learning, and is an integral step towards semantically-grounded object categorization. Whereas mainstream approaches reduce human knowledge to mere category labels on exemplars, this work leverages semantically rich knowledge more deeply and earlier in the learning pipeline. The approach results in vision systems that are less prone to overfit incidental visual patterns, and representations that are readily extendible to novel visual learning tasks. Beyond the research community, the work has broader impact through inter-disciplinary training of graduate and undergraduate students, and outreach to pre-college educators and students through workshops and summer camps encouraging young students to pursue science and engineering.