Collaborative Research: Algorithms, Theory, and Validation of Deep Graph Learning with Limited Supervision: A Continuous Perspective

协作研究：有限监督下的深度图学习的算法、理论和验证：连续的视角

• 批准号: 2208361
• 负责人: Bao Wang
• 金额: $ 24万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2208361&HistoricalAwards=false
• 关键词: Collaborative; Research; Algorithms; Theory; Validation

Graph-structured data is ubiquitous in scientific and artificial intelligence applications, for instance, particle physics, computational chemistry, drug discovery, neural science, recommender systems, robotics, social networks, and knowledge graphs. Graph neural networks (GNNs) have achieved tremendous success in a broad class of graph learning tasks, including graph node classification, graph edge prediction, and graph generation. Nevertheless, there are several bottlenecks of GNNs: 1) In contrast to many deep networks such as convolutional neural networks, it has been noticed that increasing the depth of GNNs results in a severe accuracy degradation, which has been interpreted as over-smoothing in the machine learning community. 2) The performance of GNNs relies heavily on a sufficient number of labeled graph nodes; the prediction of GNNs will become significantly less reliable when less labeled data is available. This research aims to address these challenges by developing new mathematical understanding of GNNs and theoretically-principled algorithms for graph deep learning with less training data. The project will train graduate students and postdoctoral associates through involvement in the research. The project will also integrate the research into teaching to advance data science education.This project aims to develop next-generation continuous-depth GNNs leveraging computational mathematics tools and insights and to advance data-driven scientific simulation using the new GNNs. This project has three interconnected thrusts that revolve around pushing the envelope of theory and practice in graph deep learning with limited supervision using PDE and harmonic analysis tools: 1) developing a new generation of diffusion-based GNNs that are certifiable to learning with deep architectures and less training data; 2) developing a new efficient attention-based approach for learning graph structures from the underlying data accompanied by uncertainty quantification; and 3) application validation in learning-assisted scientific simulation and multi-modal learning and software development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图结构数据在科学和人工智能应用中无处不在，例如粒子物理学、计算化学、药物发现、神经科学、推荐系统、机器人、社交网络和知识图谱。图神经网络（GNN）在广泛的图学习任务中取得了巨大的成功，包括图节点分类、图边缘预测和图生成。然而，GNN 存在几个瓶颈：1）与许多深度网络（例如卷积神经网络）相比，人们注意到增加 GNN 的深度会导致严重的精度下降，这在机器学习社区中被解释为过度平滑。 2）GNN的性能很大程度上依赖于足够数量的标记图节点；当可用的标记数据较少时，GNN 的预测将变得不太可靠。本研究旨在通过开发对 GNN 的新数学理解以及使用较少训练数据进行图深度学习的理论原理算法来应对这些挑战。该项目将通过参与研究来培训研究生和博士后。该项目还将研究融入教学，以推进数据科学教育。该项目旨在利用计算数学工具和见解开发下一代连续深度 GNN，并使用新的 GNN 推进数据驱动的科学模拟。该项目具有三个相互关联的主旨，围绕使用偏微分方程和调和分析工具在有限监督下推动图深度学习的理论和实践的极限：1）开发新一代基于扩散的 GNN，可通过深度架构和更少的训练数据进行学习； 2）开发一种新的有效的基于注意力的方法，用于从底层数据中学习图结构并伴随不确定性量化； 3）学习辅助科学模拟和多模式学习和软件开发中的应用验证。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

A deterministic gradient-based approach to avoid saddle points

一种避免鞍点的基于确定性梯度的方法

• DOI: 10.1017/s0956792522000316
• 发表时间: 2022
• 期刊: European Journal of Applied Mathematics
• 影响因子: 1.9
• 作者: Kreusser, L. M.;Osher, S. J.;Wang, B.
• 通讯作者: Wang, B.

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

• DOI: 10.1137/21m1465081
• 发表时间: 2021-12
• 期刊: SIAM J. Appl. Math.
• 作者: Yifan Hua;Kevin Miller;A. Bertozzi;Chen Qian;Bao Wang

Implicit Graph Neural Networks: A Monotone Operator Viewpoint

• 发表时间: 2023
• 作者: Justin Baker;Qingsong Wang;C. Hauck;Bao Wang

Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient Momentum

• DOI: 10.1007/s10915-023-02148-y
• 发表时间: 2022-08
• 期刊: Journal of Scientific Computing
• 影响因子: 2.5
• 作者: Mengqi Hu;Y. Lou;Bao Wang;Ming Yan;Xiu Yang;Q. Ye

How does momentum benefit deep neural networks architecture design? A few case studies

• DOI: 10.1007/s40687-022-00352-0
• 发表时间: 2021-10
• 期刊: Research in the Mathematical Sciences
• 影响因子: 1.2
• 作者: Bao Wang;Hedi Xia;T. Nguyen;S. Osher