Understanding and Improving Deep Learning for Structured Data

理解和改进结构化数据的深度学习

• 批准号: RGPIN-2022-04636
• 负责人: Liao, Renjie
• 金额: $ 2.11万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752175
• 关键词: Understanding; Improving; Deep; Learning; Structured

Deep learning has achieved tremendous success in processing data with regular structures like images (grids), languages (sequences), and speech (sequences). However, data with irregular structure (e.g., point clouds in computer vision, multi-agent interaction graphs in robotics, parsing trees in natural language processing, molecules in computational chemistry, protein structure in biology, and social networks in computational social science) is ubiquitous and poses challenges to current deep learning methods. In this research program, we aim to understand the limitations of current deep learning models for structured data and improve them accordingly. In the first part, we will study the out-of-distribution (OOD) generalization ability of graph neural networks (GNNs) and Transformers in the context of reasoning tasks. For example, in algorithmic reasoning tasks where we know correct algorithms exist, we will investigate if GNNs/Transformers can learn graph algorithms that generalize to OOD graph data. These studies would shed light on designing novel models with better inductive bias. Once we obtain conclusions from such well-defined tasks, we will move to more challenging and realistic reasoning problems, e.g., learning neural networks based solvers for NP-hard problems, visual reasoning tasks such as visual question answering and scene graph prediction, and motion planning for self-driving. This part would help improve deep learning models on reasoning tasks and improve their OOD generalization on structured data. In the second part, we plan to design novel deep generative models for structured data. Given the superior performance of the recent deep generative models on images, it is appealing to generalize them to structured data like graphs. However, we need to overcome two main challenges: 1) building expressive and permutation-invariant probabilistic models for graphs; 2) designing customized sampling and learning algorithms for discrete random variables. Moreover, we plan to extend unconditional deep generative models to the conditional setting so that we can learn latent structures from data. Such models could describe the uncertainty and help improve the interpretability, which are crucial for real-world applications like point clouds generation in 3D compute vision and molecule generation for drug discovery.

深度学习在处理具有规则结构的数据方面取得了巨大的成功，如图像（网格），语言（序列）和语音（序列）。然而，具有不规则结构的数据（例如，计算机视觉中的点云、机器人中的多代理交互图、自然语言处理中的解析树、计算化学中的分子、生物学中的蛋白质结构以及计算社会科学中的社交网络）是普遍存在的，并且对当前的深度学习方法提出了挑战。在这项研究计划中，我们的目标是了解当前结构化数据深度学习模型的局限性，并相应地改进它们。 在第一部分中，我们将研究图神经网络（GNNs）和Transformer在推理任务中的分布外（OOD）泛化能力。例如，在我们知道存在正确算法的算法推理任务中，我们将研究GNN/Transformer是否可以学习推广到OOD图数据的图算法。这些研究将有助于设计具有更好感应偏置的新型模型。一旦我们从这些定义明确的任务中得出结论，我们将转向更具挑战性和现实性的推理问题，例如，基于学习神经网络的NP难题求解器，视觉推理任务，如视觉问答和场景图预测，以及自动驾驶的运动规划。这部分将有助于改进推理任务的深度学习模型，并改进其在结构化数据上的OOD泛化。在第二部分中，我们计划为结构化数据设计新的深度生成模型。鉴于最近的深度生成模型在图像上的上级性能，将它们推广到结构化数据（如图）是很有吸引力的。然而，我们需要克服两个主要挑战：1）为图构建表达性和置换不变的概率模型; 2）为离散随机变量设计定制的采样和学习算法。此外，我们计划将无条件深度生成模型扩展到条件设置，以便我们可以从数据中学习潜在结构。这些模型可以描述不确定性并有助于提高可解释性，这对于3D计算视觉中的点云生成和药物发现的分子生成等现实应用至关重要。