EAGER: Combining Knowledge with Data for Generalizable and Robust Visual Learning

计算机视觉在过去的几十年里取得了巨大的进步， 先进的机器学习技术。 但与人类感知相比，计算机视觉仍然很原始。 造成这种情况的一个因素是当前学习算法的数据驱动性质，以及它们无法整合任何相关知识。 数据驱动的方法往往是特定于数据库的，不能很好地推广到看不见的数据。该项目通过采用知识增强型统计学习框架来解决这一问题。 在这个框架内，知识和数据可以被系统地利用，捕获，并主要集成以联合训练视觉算法。 然而，开发这样一个框架是具有挑战性的，因为领域知识通常以不同的形式存在，通常无法使用数据驱动的统计机器学习方法。 为了克服这一挑战，研究团队系统地将领域知识转换为模型的约束或伪数据，从而可以将它们纳入统计学习方法。 该项目包括系统地识别来自不同来源的知识，以及获取知识并将其转换为易于自动机器学习方法访问的格式的具体机制。 该项目还包括展示拟议框架对某些计算机视觉问题的有效性。该项目为研究生和本科生提供培训，并通过出版物和组织相关研讨会传播研究成果。

Computer vision has made tremendous progress in the past decades, partially enabled by the advanced machine learning techniques. But compared with human perception, computer vision remains primitive. One contributing factor for this is the data-driven nature of the current learning algorithms and their inability to incorporate any related knowledge. The data-driven methods tend to be database-specific and cannot generalize well to unseen data. This project addresses this issue through the introduction of a knowledge-augmented statistical learning framework. Within this framework, knowledge and data can be systematically exploited, captured, and are principally integrated to jointly train a vision algorithm. Developing such a framework, however, is challenging since the domain knowledge often exists in different and diverse formats, typically inaccessible to the data-driven statistical machine learning methods. To overcome this challenge, the research team systematically converts domain knowledge into either the constraints on the model or into pseudo-data, whereby they can be incorporated into the statistical learning methods. The project includes systematic identification of knowledge from different sources and concrete mechanisms to capture the knowledge and to convert them into formats easily accessible to the automatic machine learning methods. The project also involves demonstrating the effectiveness of the proposed framework for certain computer vision problems.The project provides the training for graduate and undergraduate students, and the research results are disseminated through publications and organization of the related workshops.