Using structural priors to learn transferable representations

背景：近年来，深度学习在计算机视觉、自然语言处理和其他核心人工智能领域的几项任务上取得了令人印象深刻的进展。这导致了对更复杂和更现实的应用的更多努力，通常涉及多种模式(例如图像和文本)。机器学习从业者面临的一个挑战是，通常经过训练的模型在新数据上表现不佳。特别是，分布外(OOD)泛化问题，其中假设新数据不是来自与训练数据相同的分布。例如，在医学成像方面，一个模型可以根据几家医院的数据进行训练，但将用于分析来自其他医院的图像，这些医院使用不同的设备和患者分布。这个问题是由于模型拘泥于训练数据中的虚假相关性或“捷径”而引起的，这些关联或“捷径”并不对应于有意义的解决方案。目的：我们的研究计划致力于开发灵活的体系结构，以提高现实任务(例如，医学成像中的患者，机器人中的模拟到真实的机器人，或遥感中不同地区的卫星图像)中的面向对象设计的普适性。为了能够开发和测试新的面向对象设计泛化体系结构，我们的短期目标集中在以下三个领域：(I)在多模学习中学习解缠表示，(Ii)在强化学习代理中学习泛化技能，重点是捕获动力学，以及(Iii)少机会学习作为多模特征可转移性的试验床。以类似的方法在所有这些领域取得进展将是朝着长期目标迈出的重要一步。在每个领域，我们将研究两组体系结构先验：(I)模式之间的知识转移(例如，知识蒸馏或特征调制)和(Ii)层次表示(例如，涉及乘性交互的结构化模型体系结构)。意义：较差的面向对象设计泛化是在许多实际任务中部署深度学习方法的障碍。解决这个问题需要捕获模型中的域结构。可以利用多个模式和来自领域专家的先验知识的体系结构可以被引导到更通用的解决方案，从而避免了当前体系结构学到的许多捷径。拟议研究的潜在影响是深远的，因为快捷学习使模型容易反映多种形式的偏见，这些偏见是数据集或常用方法的体系结构细节中固有的。减少偏见保证了改进个性化解决方案的灵活性，例如在医疗诊断或教育方面。在数据收集昂贵的领域(例如医学成像、机器人、高分辨率遥感)，可以使用较少的例子来训练结构化模型。

General context: In recent years, deep learning has enabled impressive progress on several tasks in computer vision, natural language processing, and other core artificial intelligence domains. This has led to more effort towards more complex and realistic applications, often involving multiple modalities (e.g. images and text). One challenge machine learning practitioners face is that often trained models do not perform well on new data. In particular, the problem of out-of-distribution (OOD) generalization, where the assumption is that new data is not drawn from the same distribution as the training data. For instance, in medical imaging, a model could be trained on data from several hospitals but will be used to analyze images from other hospitals with different devices and patient distributions. This problem is caused by models latching onto spurious correlations or "shortcuts" in the training data, which do not correspond to a meaningful solution. Objectives: Our research program is concerned with the development of flexible architectural priors that improve OOD generalization in real-world tasks (e.g. across patients in medical imaging, from simulation to a real robot in robotics, or across satellite images from different regions in remote sensing). To be able to develop and test novel architectures for OOD generalization, we focus on the following three domains in our short-term goals: (i) learning of disentangled representations in multimodal learning, (ii) learning generalizable skills in reinforcement learning agents with a focus on capturing dynamics, and (iii) few-shot learning as a test-bed for transferability of multimodal features. Progress on all of these domains with a similar methodology will be a significant step towards the long-term objective. In each of these domains, we will investigate two groups of architectural priors: (i) knowledge transfer between modalities (e.g. knowledge distillation or feature modulation) and (ii) hierarchical representations (e.g. structured model architectures involving multiplicative interactions). Significance: Poor OOD generalization is an obstacle in the deployment of deep learning methods on many real-world tasks. Solving the problem requires capturing domain structure in the model. An architecture that can leverage multiple modalities and prior knowledge from domain experts, can be guided towards a more general solution, avoiding many of the shortcuts learned by current architectures. The potential impact of the proposed research is profound since shortcut learning makes models prone to reflecting many forms of bias, that are inherent in datasets or architectural details of commonly-used methods. Reducing bias promises the flexibility of improving personalized solutions, such as in medical diagnostics or education. In domains where data collection is expensive (e.g. medical imaging, robotics, high-resolution remote sensing), structured models can be trained using fewer examples.