Collaborative Research: Algorithms, Theory, and Validation of Deep Graph Learning with Limited Supervision: A Continuous Perspective

协作研究：有限监督下的深度图学习的算法、理论和验证：连续的视角

• 批准号: 2208272
• 负责人: Stanley Osher
• 金额: $ 28万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2208272&HistoricalAwards=false
• 关键词: Collaborative; Research; Algorithms; Theory; Validation

Graph-structured data is ubiquitous in scientific and artificial intelligence applications, for instance, particle physics, computational chemistry, drug discovery, neural science, recommender systems, robotics, social networks, and knowledge graphs. Graph neural networks (GNNs) have achieved tremendous success in a broad class of graph learning tasks, including graph node classification, graph edge prediction, and graph generation. Nevertheless, there are several bottlenecks of GNNs: 1) In contrast to many deep networks such as convolutional neural networks, it has been noticed that increasing the depth of GNNs results in a severe accuracy degradation, which has been interpreted as over-smoothing in the machine learning community. 2) The performance of GNNs relies heavily on a sufficient number of labeled graph nodes; the prediction of GNNs will become significantly less reliable when less labeled data is available. This research aims to address these challenges by developing new mathematical understanding of GNNs and theoretically-principled algorithms for graph deep learning with less training data. The project will train graduate students and postdoctoral associates through involvement in the research. The project will also integrate the research into teaching to advance data science education.This project aims to develop next-generation continuous-depth GNNs leveraging computational mathematics tools and insights and to advance data-driven scientific simulation using the new GNNs. This project has three interconnected thrusts that revolve around pushing the envelope of theory and practice in graph deep learning with limited supervision using PDE and harmonic analysis tools: 1) developing a new generation of diffusion-based GNNs that are certifiable to learning with deep architectures and less training data; 2) developing a new efficient attention-based approach for learning graph structures from the underlying data accompanied by uncertainty quantification; and 3) application validation in learning-assisted scientific simulation and multi-modal learning and software development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图结构数据在科学和人工智能应用中无处不在，例如粒子物理学、计算化学、药物发现、神经科学、推荐系统、机器人技术、社交网络和知识图。图神经网络（GNN）在广泛的图学习任务中取得了巨大的成功，包括图节点分类，图边缘预测和图生成。尽管如此，GNN仍存在几个瓶颈：1）与许多深度网络（如卷积神经网络）相比，人们注意到，增加GNN的深度会导致严重的准确性下降，这在机器学习社区中被解释为过度平滑。2)GNN的性能在很大程度上依赖于足够数量的标记图节点;当可用的标记数据较少时，GNN的预测将变得不那么可靠。这项研究旨在通过开发对GNN的新的数学理解和理论上的算法来解决这些挑战，以便在训练数据较少的情况下进行图深度学习。该项目将通过参与研究来培训研究生和博士后助理。该项目还将把研究融入教学，以推进数据科学教育。该项目旨在利用计算数学工具和见解开发下一代连续深度GNN，并使用新GNN推进数据驱动的科学模拟。该项目有三个相互关联的目标，围绕着使用PDE和谐波分析工具在有限监督下推动图深度学习的理论和实践：1）开发新一代基于扩散的GNN，这些GNN可通过深度架构和更少的训练数据进行学习; 2）开发一种新的有效的基于注意力的方法，用于从伴随着不确定性量化的底层数据中学习图结构;以及3）在学习辅助科学模拟和多模式学习以及软件开发中的应用验证。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估而被认为值得支持。

项目成果

A Hamilton-Jacobi-based proximal operator.

• DOI: 10.1073/pnas.2220469120
• 发表时间: 2023-04-04
• 期刊: PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
• 影响因子: 11.1
• 作者: Osher, Stanley;Heaton, Howard;Fung, Samy Wu
• 通讯作者: Fung, Samy Wu

A Primal-Dual Framework for Transformers and Neural Networks

• 发表时间: 2024-06
• 期刊: Exploration of Immunology
• 作者: T. Nguyen;Tam Nguyen;Nhat Ho;A. Bertozzi;Richard Baraniuk;S. Osher