Understanding and Improving Deep Learning for Structured Data

点击翻译按钮获取中文摘要

Deep learning has achieved tremendous success in processing data with regular structures like images (grids), languages (sequences), and speech (sequences). However, data with irregular structure (e.g., point clouds in computer vision, multi-agent interaction graphs in robotics, parsing trees in natural language processing, molecules in computational chemistry, protein structure in biology, and social networks in computational social science) is ubiquitous and poses challenges to current deep learning methods. In this research program, we aim to understand the limitations of current deep learning models for structured data and improve them accordingly. In the first part, we will study the out-of-distribution (OOD) generalization ability of graph neural networks (GNNs) and Transformers in the context of reasoning tasks. For example, in algorithmic reasoning tasks where we know correct algorithms exist, we will investigate if GNNs/Transformers can learn graph algorithms that generalize to OOD graph data. These studies would shed light on designing novel models with better inductive bias. Once we obtain conclusions from such well-defined tasks, we will move to more challenging and realistic reasoning problems, e.g., learning neural networks based solvers for NP-hard problems, visual reasoning tasks such as visual question answering and scene graph prediction, and motion planning for self-driving. This part would help improve deep learning models on reasoning tasks and improve their OOD generalization on structured data. In the second part, we plan to design novel deep generative models for structured data. Given the superior performance of the recent deep generative models on images, it is appealing to generalize them to structured data like graphs. However, we need to overcome two main challenges: 1) building expressive and permutation-invariant probabilistic models for graphs; 2) designing customized sampling and learning algorithms for discrete random variables. Moreover, we plan to extend unconditional deep generative models to the conditional setting so that we can learn latent structures from data. Such models could describe the uncertainty and help improve the interpretability, which are crucial for real-world applications like point clouds generation in 3D compute vision and molecule generation for drug discovery.