Using structural priors to learn transferable representations

点击翻译按钮获取中文摘要

General context: In recent years, deep learning has enabled impressive progress on several tasks in computer vision, natural language processing, and other core artificial intelligence domains. This has led to more effort towards more complex and realistic applications, often involving multiple modalities (e.g. images and text). One challenge machine learning practitioners face is that often trained models do not perform well on new data. In particular, the problem of out-of-distribution (OOD) generalization, where the assumption is that new data is not drawn from the same distribution as the training data. For instance, in medical imaging, a model could be trained on data from several hospitals but will be used to analyze images from other hospitals with different devices and patient distributions. This problem is caused by models latching onto spurious correlations or "shortcuts" in the training data, which do not correspond to a meaningful solution. Objectives: Our research program is concerned with the development of flexible architectural priors that improve OOD generalization in real-world tasks (e.g. across patients in medical imaging, from simulation to a real robot in robotics, or across satellite images from different regions in remote sensing). To be able to develop and test novel architectures for OOD generalization, we focus on the following three domains in our short-term goals: (i) learning of disentangled representations in multimodal learning, (ii) learning generalizable skills in reinforcement learning agents with a focus on capturing dynamics, and (iii) few-shot learning as a test-bed for transferability of multimodal features. Progress on all of these domains with a similar methodology will be a significant step towards the long-term objective. In each of these domains, we will investigate two groups of architectural priors: (i) knowledge transfer between modalities (e.g. knowledge distillation or feature modulation) and (ii) hierarchical representations (e.g. structured model architectures involving multiplicative interactions). Significance: Poor OOD generalization is an obstacle in the deployment of deep learning methods on many real-world tasks. Solving the problem requires capturing domain structure in the model. An architecture that can leverage multiple modalities and prior knowledge from domain experts, can be guided towards a more general solution, avoiding many of the shortcuts learned by current architectures. The potential impact of the proposed research is profound since shortcut learning makes models prone to reflecting many forms of bias, that are inherent in datasets or architectural details of commonly-used methods. Reducing bias promises the flexibility of improving personalized solutions, such as in medical diagnostics or education. In domains where data collection is expensive (e.g. medical imaging, robotics, high-resolution remote sensing), structured models can be trained using fewer examples.