Collaborative Research: Algorithms, Theory, and Validation of Deep Graph Learning with Limited Supervision: A Continuous Perspective

图结构数据在科学和人工智能应用中无处不在，例如粒子物理、计算化学、药物发现、神经科学、推荐系统、机器人、社交网络和知识图。图神经网络（gnn）在广泛的图学习任务中取得了巨大的成功，包括图节点分类、图边缘预测和图生成。然而，gnn存在几个瓶颈：1)与卷积神经网络等许多深度网络相比，人们已经注意到，增加gnn的深度会导致严重的精度下降，这在机器学习社区中被解释为过度平滑。2) gnn的性能严重依赖于足够数量的标记图节点；当可用的标记数据较少时，gnn的预测将变得明显不可靠。本研究旨在通过开发对gnn的新的数学理解和具有较少训练数据的图深度学习的理论原则算法来解决这些挑战。该项目将通过参与研究培养研究生和博士后。该项目还将把研究整合到教学中，以推进数据科学教育。该项目旨在利用计算数学工具和见解开发下一代连续深度gnn，并使用新的gnn推进数据驱动的科学模拟。该项目有三个相互关联的重点，围绕着使用PDE和谐波分析工具在有限监督下推动图深度学习的理论和实践：1)开发新一代基于扩散的gnn，该gnn可以通过深度架构和更少的训练数据进行学习；2)开发一种新的高效的基于注意力的方法，从底层数据中学习图结构，并伴有不确定性量化；3)在辅助学习科学仿真和多模态学习及软件开发中的应用验证。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Graph-structured data is ubiquitous in scientific and artificial intelligence applications, for instance, particle physics, computational chemistry, drug discovery, neural science, recommender systems, robotics, social networks, and knowledge graphs. Graph neural networks (GNNs) have achieved tremendous success in a broad class of graph learning tasks, including graph node classification, graph edge prediction, and graph generation. Nevertheless, there are several bottlenecks of GNNs: 1) In contrast to many deep networks such as convolutional neural networks, it has been noticed that increasing the depth of GNNs results in a severe accuracy degradation, which has been interpreted as over-smoothing in the machine learning community. 2) The performance of GNNs relies heavily on a sufficient number of labeled graph nodes; the prediction of GNNs will become significantly less reliable when less labeled data is available. This research aims to address these challenges by developing new mathematical understanding of GNNs and theoretically-principled algorithms for graph deep learning with less training data. The project will train graduate students and postdoctoral associates through involvement in the research. The project will also integrate the research into teaching to advance data science education.This project aims to develop next-generation continuous-depth GNNs leveraging computational mathematics tools and insights and to advance data-driven scientific simulation using the new GNNs. This project has three interconnected thrusts that revolve around pushing the envelope of theory and practice in graph deep learning with limited supervision using PDE and harmonic analysis tools: 1) developing a new generation of diffusion-based GNNs that are certifiable to learning with deep architectures and less training data; 2) developing a new efficient attention-based approach for learning graph structures from the underlying data accompanied by uncertainty quantification; and 3) application validation in learning-assisted scientific simulation and multi-modal learning and software development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.