Using structural priors to learn transferable representations

一般情况：近年来，深度学习在计算机视觉、自然语言处理和其他核心人工智能领域的多项任务上取得了令人印象深刻的进展。这导致更多的努力，以更复杂和现实的应用程序，往往涉及多种形式（如图像和文本）。机器学习从业者面临的一个挑战是，通常经过训练的模型在新数据上表现不佳。特别是，分布外（OOD）泛化的问题，其中假设新数据不是从与训练数据相同的分布中提取的。例如，在医学成像中，模型可以在来自几家医院的数据上进行训练，但将用于分析来自其他医院的图像，这些医院具有不同的设备和患者分布。这个问题是由模型锁定训练数据中的虚假相关性或“捷径”引起的，这与有意义的解决方案不相对应。目的：我们的研究计划关注灵活的建筑先验的发展，提高面向对象的泛化在现实世界的任务（例如，在医疗成像患者，从模拟到机器人技术中的真实的机器人，或在遥感不同地区的卫星图像）。为了能够开发和测试OOD泛化的新架构，我们在短期目标中专注于以下三个领域：（i）在多模态学习中学习解纠缠表示，（ii）在强化学习代理中学习可泛化的技能，重点是捕获动态，以及（iii）作为多模态特征可转移性测试平台的少数学习。在所有这些领域采用类似的方法取得进展，将是朝着实现长期目标迈出的重要一步。 在这些领域中，我们将研究两组架构先验：（i）模态之间的知识转移（例如知识蒸馏或特征调制）和（ii）分层表示（例如涉及乘法交互的结构化模型架构）。意义：糟糕的OOD泛化是在许多现实任务中部署深度学习方法的障碍。解决这个问题需要在模型中捕获域结构。一个可以利用多个模态和领域专家的先验知识的架构，可以被引导到一个更通用的解决方案，避免当前架构所学到的许多捷径。所提出的研究的潜在影响是深远的，因为捷径学习使模型容易反映出许多形式的偏见，这些偏见是数据集或常用方法的架构细节所固有的。减少偏差保证了改进个性化解决方案的灵活性，例如在医疗诊断或教育方面。在数据收集昂贵的领域（例如医学成像，机器人，高分辨率遥感），结构化模型可以使用更少的示例进行训练。

General context: In recent years, deep learning has enabled impressive progress on several tasks in computer vision, natural language processing, and other core artificial intelligence domains. This has led to more effort towards more complex and realistic applications, often involving multiple modalities (e.g. images and text). One challenge machine learning practitioners face is that often trained models do not perform well on new data. In particular, the problem of out-of-distribution (OOD) generalization, where the assumption is that new data is not drawn from the same distribution as the training data. For instance, in medical imaging, a model could be trained on data from several hospitals but will be used to analyze images from other hospitals with different devices and patient distributions. This problem is caused by models latching onto spurious correlations or "shortcuts" in the training data, which do not correspond to a meaningful solution. Objectives: Our research program is concerned with the development of flexible architectural priors that improve OOD generalization in real-world tasks (e.g. across patients in medical imaging, from simulation to a real robot in robotics, or across satellite images from different regions in remote sensing). To be able to develop and test novel architectures for OOD generalization, we focus on the following three domains in our short-term goals: (i) learning of disentangled representations in multimodal learning, (ii) learning generalizable skills in reinforcement learning agents with a focus on capturing dynamics, and (iii) few-shot learning as a test-bed for transferability of multimodal features. Progress on all of these domains with a similar methodology will be a significant step towards the long-term objective. In each of these domains, we will investigate two groups of architectural priors: (i) knowledge transfer between modalities (e.g. knowledge distillation or feature modulation) and (ii) hierarchical representations (e.g. structured model architectures involving multiplicative interactions). Significance: Poor OOD generalization is an obstacle in the deployment of deep learning methods on many real-world tasks. Solving the problem requires capturing domain structure in the model. An architecture that can leverage multiple modalities and prior knowledge from domain experts, can be guided towards a more general solution, avoiding many of the shortcuts learned by current architectures. The potential impact of the proposed research is profound since shortcut learning makes models prone to reflecting many forms of bias, that are inherent in datasets or architectural details of commonly-used methods. Reducing bias promises the flexibility of improving personalized solutions, such as in medical diagnostics or education. In domains where data collection is expensive (e.g. medical imaging, robotics, high-resolution remote sensing), structured models can be trained using fewer examples.