Collaborative Research: Theory and Applications of Structure-Conforming Deep Operator Learning

基于第一性原理的方法在许多工程和科学学科中取得了相当大的成功，包括流体和固体力学、电磁学等。其中最重要的应用是偏微分方程（PDEs），它与它们的分析和数值算法相结合，代表了人类为理解物质世界而开发的一些最强大的工具。然而，从物理学、生物学和化学中产生的日益复杂的数学模型对基于第一性原理的方法解决实际问题的有效性提出了挑战，例如流体湍流、分子动力学和大规模逆问题。数值算法的一个主要障碍是所谓的“维数诅咒”。在图形处理单元（Graphics Processing Unit）和张量处理单元（Tensor Processing Unit）通用计算技术进步的推动下，深度神经网络（dnn）和深度学习方法在对抗维度诅咒方面表现出色，并在解决科学和工程中的复杂问题方面显示出巨大的潜力。该项目旨在研究问题中的数学结构如何为创新dnn的设计和分析提供信息，特别是在逆问题的背景下，从测量中推断出未知参数，如电阻抗断层扫描。此外，这个项目的编程部分将侧重于培养下一代计算数学家。深度学习中的算子学习（OpL）框架为解决具有挑战性和潜在不适定的基于pde的问题提供了独特的视角。该项目将探索OpL的潜力，以减轻许多逆问题的病态性，因为其强大的近似能力与离线训练和在线预测特性相结合，可以实现高质量，快速的重建。该项目旨在通过将经典问题解决方法中的数学结构集成到深度神经网络架构中，从而将OpL和经典方法连接起来。特别是，该项目将揭示注意力机制的数学性质，这是GPT和AlphaFold 2等最先进的深度神经网络变压器的支柱。此外，该项目将研究注意力神经结构的灵活性，根据问题的先验数学结构，将注意力机制与应用数学中的重要方法（如伽辽金投影或Fredholm积分方程）融合在一起。该项目还将通过希尔伯特空间的光谱理论深入研究注意力的数学基础，试图理解象征性的查询-键-值架构如何有助于变形金刚丰富的表征能力和多样化的近似能力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The first-principle-based approach has achieved considerable success in numerous engineering and scientific disciplines, including fluid and solid mechanics, electromagnetism, and more. Among its most significant applications are partial differential equations (PDEs) which, in conjunction with their analysis and numerical algorithms, represent some of the most powerful tools humanity has ever developed for understanding the material world. However, increasingly complex mathematical models arising from physics, biology, and chemistry challenge the efficacy of first-principle-based approaches for solving practical problems, such as those in fluid turbulence, molecular dynamics, and large-scale inverse problems. A major obstacle for numerical algorithms is the so-called curse of dimensionality. Fueled by advances in Graphics Processing Unit and Tensor Processing Unit general-purpose computing, deep neural networks (DNNs) and deep learning approaches excel in combating the curse of dimensionality and demonstrate immense potential for solving complex problems in science and engineering. This project aims to investigate how mathematical structures within a problem can inform the design and analysis of innovative DNNs, particularly in the context of inverse problems where unknown parameters are inferred from measurements, such as electrical impedance tomography. Additionally, the programming component in this project will focus on training the next generation of computational mathematicians.The Operator Learning (OpL) framework in deep learning provides a unique perspective for tackling challenging and potentially ill-posed PDE-based problems. This project will explore the potential of OpL to mitigate the ill-posedness of many inverse problems, as its powerful approximation capability combined with offline training and online prediction properties lead to high-quality, rapid reconstructions. The project seeks to bridge OpL and classical methodologies by integrating mathematical structures from classical problem-solving approaches into DNN architectures. In particular, the project will shed light on the mathematical properties of the attention mechanism, the backbone of state-of-the-art DNN Transformers, such as those in GPT and AlphaFold 2. Furthermore, the project will examine the flexibility of attention neural architectures, enabling the fusion of attention mechanisms with important methodologies in applied mathematics, such as Galerkin projection or Fredholm integral equations, in accordance with the a priori mathematical structure of a problem. This project will also delve into the mathematical foundations of attention through the lens of spectral theory in Hilbert spaces, seeking to understand how the emblematic query-key-value architecture contributes to the rich representational power and diverse approximation capabilities of Transformers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.