Expressivity of Structure-Preserving Deep Neural Networks for the Space-Time Approximation of High-Dimensional Nonlinear Partial Differential Equations with Boundaries

偏微分方程(PDE)是在对许多自然、工业和金融现象进行建模和研究时自然产生的。虽然存在许多逼近多种类型偏微分方程组的解的技术，但实际感兴趣的某些问题表现出各种类型的病理，这使得许多标准技术要么效率低下，要么不可用。这种情况经常出现在高维或表现出奇异性的非线性问题的背景下，例如在固体燃料燃烧优化研究、输油管道腐蚀预测和高频金融交易中。该项目旨在通过严格研究和开发所谓的结构保持深度神经网络(DNN)来提供规避上述问题的手段。虽然已经在实验中观察到DNN提供了非常有能力的方法来近似解决一大类问题，但事实是，这种观察的理论证明仍然处于早期阶段。为此，本项目将通过双管齐下的方法为某些类型的高维非线性偏微分方程组提供急需的理论：即将构建显式随机化方法，展示所需的性质，同时开发通过DNN表示和研究大类对象的理论工具。这种方法需要使用应用数学、泛函分析、随机分析和新颖的DNN计算中的许多工具。这一独特的技术交叉将作为该项目教育和培训部分的基础，旨在增加妇女、少数群体和其他代表性不足的群体在数学研究中的存在。这一目标将通过在研究生和本科阶段对第一代和代表性不足的学生进行培训和指导来实现。本项目旨在解决以下问题：是否可以严格证明存在DNN来近似求解一大类高维PDE而不受维度诅咒(CoD)的影响。证明了DNN能够以规定的精度表示某些类型的高维非线性偏微分方程组的解，而不受CoD的影响，这将填补现有机器学习算法理论中的一个空白。虽然目前的重点是研究DNN对偏微分方程组的解的表达能力，但这项工作也将作为将此类研究扩展到其他类型问题的基础。这个项目将把现有的多层Picard(MLP)逼近方法的理论扩展到更一般的高维非线性偏微分方程组，专注于保持固有的定性结构。对MLP逼近的研究也将导致关于各种随机不动点方程的新的理论结果。最后，提出的工作将提供关于如何构造各种数学对象的无CoD的DNN表示的明确细节，同时也探索与流行的激活函数相关的理论问题。这些想法将被用来证明DNN-表示结果，并将提供对激活功能如何影响优化的更深层次的理解。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Partial differential equations (PDEs) arise naturally in the modeling and study of many natural, industrial, and financial phenomena. While there exist numerous techniques for approximating solutions to many types of PDEs, it is the case that certain problems of practical interest exhibit various types of pathology, which render many standard techniques either highly inefficient or unusable. Such situations often arise in the setting of nonlinear problems which are posed in high dimensions or exhibit singularities, such as in the study of solid-fuel combustion optimization, oil pipeline corrosion predictions, and high-frequency financial trading. This project intends to provide means of circumventing the aforementioned issues through the rigorous study and development of so-called structure-preserving deep neural networks (DNNs). While it has been experimentally observed that DNNs provide highly capable methods of approximating solutions to a large class of problems, it is the case that the theoretical justification of such observations is still in its early stage. To that end, this project will provide a much-needed theory for certain classes of high-dimensional nonlinear PDEs via a two-pronged approach: namely, explicit randomized methods will be constructed which demonstrate desired properties while simultaneously developing theoretical tools representing and studying a large class of objects via DNNs. This approach requires the use of numerous tools from applied mathematics, functional analysis, stochastic analysis, and novel DNN computations. This unique intersection of techniques will serve as the basis for the project's educational and training components, which aim to increase the presence of women, minorities, and other underrepresented groups in mathematical research. This goal will be accomplished through the training and mentoring first-generation and underrepresented students at both graduate and undergraduate levels.This project aims to address the question of whether or not it can be rigorously proven that there exist DNNs to approximate solutions to a large class of high-dimensional PDEs without suffering from the curse of dimensionality (CoD). The demonstration that DNNs are able to represent solutions to certain classes of high-dimensional nonlinear PDEs with a prescribed accuracy while not suffering from the CoD will fill a gap in the existing theory regarding machine learning algorithms. While the current focus is on studying the expressibility of DNNs with regard to solutions of PDEs, it is the case that this work will also serve as the foundation for extending such studies to other types of problems. This project will extend the existing theory of multilevel Picard (MLP) approximation methods to more general high-dimensional nonlinear PDEs, focusing on preserving inherent qualitative structures. The study of MLP approximations will also result in novel theoretical results regarding various stochastic fixed-point equations. Finally, the proposed work will provide explicit details on how to construct CoD-free DNN representations of various mathematical objects while also exploring theoretical issues related to popular activation functions. These ideas will be utilized in proving DNN-representation results and will provide a deeper understanding of how activation functions affect optimality.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.