Collaborative Research: Algorithms, Theory, and Validation of Deep Graph Learning with Limited Supervision: A Continuous Perspective

图结构数据在科学和人工智能应用中普遍存在，例如，粒子物理、计算化学、药物发现、神经科学、推荐系统、机器人、社交网络和知识图谱。图神经网络(GNN)在包括图节点分类、图边预测和图生成在内的一大类图学习任务中取得了巨大的成功。然而，GNN仍然存在几个瓶颈：1)与许多深层网络(如卷积神经网络)相比，人们已经注意到，增加GNN的深度会导致严重的精度下降，这在机器学习领域被解释为过度平滑。2)GNN的性能在很大程度上依赖于足够数量的标记图节点；当可用标记数据较少时，GNN的预测将变得明显不可靠。这项研究旨在通过发展对GNN的新的数学理解和理论上的原则性算法来解决这些挑战，以使用更少的训练数据进行图的深度学习。该项目将通过参与研究来培训研究生和博士后助理。该项目还将把研究融入教学，以推进数据科学教育。该项目旨在利用计算数学工具和见解开发下一代连续深度全球网络，并利用新的全球网络推进数据驱动的科学模拟。这个项目有三个相互关联的项目，围绕着利用PDE和调和分析工具在有限监督下推动图形深度学习的理论和实践的极限：1)开发新一代基于扩散的GNN，它可以通过深度体系结构和更少的训练数据进行学习；2)开发一种新的基于注意力的方法，用于从伴随着不确定性量化的底层数据学习图形结构；以及3)在学习辅助科学模拟、多模式学习和软件开发中的应用验证。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Graph-structured data is ubiquitous in scientific and artificial intelligence applications, for instance, particle physics, computational chemistry, drug discovery, neural science, recommender systems, robotics, social networks, and knowledge graphs. Graph neural networks (GNNs) have achieved tremendous success in a broad class of graph learning tasks, including graph node classification, graph edge prediction, and graph generation. Nevertheless, there are several bottlenecks of GNNs: 1) In contrast to many deep networks such as convolutional neural networks, it has been noticed that increasing the depth of GNNs results in a severe accuracy degradation, which has been interpreted as over-smoothing in the machine learning community. 2) The performance of GNNs relies heavily on a sufficient number of labeled graph nodes; the prediction of GNNs will become significantly less reliable when less labeled data is available. This research aims to address these challenges by developing new mathematical understanding of GNNs and theoretically-principled algorithms for graph deep learning with less training data. The project will train graduate students and postdoctoral associates through involvement in the research. The project will also integrate the research into teaching to advance data science education.This project aims to develop next-generation continuous-depth GNNs leveraging computational mathematics tools and insights and to advance data-driven scientific simulation using the new GNNs. This project has three interconnected thrusts that revolve around pushing the envelope of theory and practice in graph deep learning with limited supervision using PDE and harmonic analysis tools: 1) developing a new generation of diffusion-based GNNs that are certifiable to learning with deep architectures and less training data; 2) developing a new efficient attention-based approach for learning graph structures from the underlying data accompanied by uncertainty quantification; and 3) application validation in learning-assisted scientific simulation and multi-modal learning and software development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.