EAGER: Combining Knowledge with Data for Generalizable and Robust Visual Learning

点击翻译按钮获取中文摘要

Computer vision has made tremendous progress in the past decades, partially enabled by the advanced machine learning techniques. But compared with human perception, computer vision remains primitive. One contributing factor for this is the data-driven nature of the current learning algorithms and their inability to incorporate any related knowledge. The data-driven methods tend to be database-specific and cannot generalize well to unseen data. This project addresses this issue through the introduction of a knowledge-augmented statistical learning framework. Within this framework, knowledge and data can be systematically exploited, captured, and are principally integrated to jointly train a vision algorithm. Developing such a framework, however, is challenging since the domain knowledge often exists in different and diverse formats, typically inaccessible to the data-driven statistical machine learning methods. To overcome this challenge, the research team systematically converts domain knowledge into either the constraints on the model or into pseudo-data, whereby they can be incorporated into the statistical learning methods. The project includes systematic identification of knowledge from different sources and concrete mechanisms to capture the knowledge and to convert them into formats easily accessible to the automatic machine learning methods. The project also involves demonstrating the effectiveness of the proposed framework for certain computer vision problems.The project provides the training for graduate and undergraduate students, and the research results are disseminated through publications and organization of the related workshops.