Collaborative Research: Differential Equations Motivated Multi-Agent Sequential Deep Learning: Algorithms, Theory, and Validation

协作研究：微分方程驱动的多智能体序列深度学习：算法、理论和验证

• 批准号: 2152762
• 负责人: Bao Wang
• 金额: $ 10万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2152762&HistoricalAwards=false
• 关键词: Collaborative; Research; Differential; Equations; Motivated

Sequential data observed from multiple agents is ubiquitous in artificial intelligence (AI) and scientific applications, for example, in computer vision, natural language processing, robotics, computational biology and biophysics, and knowledge graphs. Learning from sequentially observed data often provides a global understanding of the underlying system and yields more reliable predictions than learning from a non-sequentially (single-shot) observed data. Sequential data is often irregularly-sampled in time and space and when this is combined with the interaction between agents, it raises tremendous challenges for machine learning. This project addresses these challenges by developing new mathematical understandings of these bottlenecks combined with new mathematically-principled deep learning algorithms for sequential and graph learning. Anticipated results and algorithms from this project will have broad applicability to important societal issues, such as pandemic spread, cooperative robotics, and environmental change. The project includes research training opportunities for graduate students.This project bridges ordinary differential equations (ODEs) and partial differential equations (PDEs) theory with multi-agent sequential learning practice. The project further leverages ODE and PDE insights to advance theoretically-grounded algorithms for deep sequential and graph learning. This project synergistically integrates recent advances in neural ODE methods with recent advances in graph networks for machine learning. The project develops and explores building next-generation algorithms based on wave equations on graphs, coupling second-order continuous dynamics in time with graph filtering. The research includes theoretical guarantees for the new methods in overcoming the over-smoothing issue, to enable sequential learning on graphs with deep architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从多个代理观察到的顺序数据在人工智能（AI）和科学应用中无处不在，例如，在计算机视觉，自然语言处理，机器人，计算生物学和生物物理学以及知识图中。从连续观测数据中学习通常提供对底层系统的全局理解，并且比从非连续（单次）观测数据中学习产生更可靠的预测。序列数据通常在时间和空间上不规则地采样，当这与代理之间的交互相结合时，它给机器学习带来了巨大的挑战。该项目通过开发对这些瓶颈的新的数学理解，结合用于顺序和图学习的新的基于数学原理的深度学习算法来解决这些挑战。该项目的预期结果和算法将广泛适用于重要的社会问题，如流行病传播，合作机器人和环境变化。该项目包括研究生的研究培训机会。该项目将常微分方程（ODE）和偏微分方程（PDE）理论与多智能体顺序学习实践联系起来。该项目进一步利用ODE和PDE见解来推进深度序列和图学习的理论基础算法。该项目将神经ODE方法的最新进展与用于机器学习的图网络的最新进展协同集成。该项目开发和探索基于图形上的波动方程构建下一代算法，将二阶连续动态与图形滤波耦合在一起。 该研究包括克服过度平滑问题的新方法的理论保证，以实现对具有深度架构的图的顺序学习。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估而被认为值得支持。

项目成果

Implicit Graph Neural Networks: A Monotone Operator Viewpoint

• 发表时间: 2023
• 作者: Justin Baker;Qingsong Wang;C. Hauck;Bao Wang

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

• DOI: 10.1137/21m1465081
• 发表时间: 2021-12
• 期刊: SIAM J. Appl. Math.
• 作者: Yifan Hua;Kevin Miller;A. Bertozzi;Chen Qian;Bao Wang

Learning Proper Orthogonal Decomposition of Complex Dynamics Using Heavy-ball Neural ODEs

使用重球神经常微分方程学习复杂动力学的正确正交分解

• DOI: 10.1007/s10915-023-02176-8
• 发表时间: 2023
• 期刊: Journal of Scientific Computing
• 影响因子: 2.5
• 作者: Baker, Justin;Cherkaev, Elena;Narayan, Akil;Wang, Bao
• 通讯作者: Wang, Bao

How does momentum benefit deep neural networks architecture design? A few case studies

• DOI: 10.1007/s40687-022-00352-0
• 发表时间: 2021-10
• 期刊: Research in the Mathematical Sciences
• 影响因子: 1.2
• 作者: Bao Wang;Hedi Xia;T. Nguyen;S. Osher

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

使用非线性共轭梯度式自适应动量改进深度神经网络 - 图像分类训练

• DOI: 10.1109/tnnls.2023.3255783
• 发表时间: 2023
• 期刊: IEEE Transactions on Neural Networks and Learning Systems
• 影响因子: 10.4
• 作者: Wang, Bao;Ye, Qiang
• 通讯作者: Ye, Qiang