Expressivity of Structure-Preserving Deep Neural Networks for the Space-Time Approximation of High-Dimensional Nonlinear Partial Differential Equations with Boundaries

保结构深度神经网络的表达能力用于高维非线性有边界偏微分方程的时空逼近

• 批准号: 2206675
• 负责人: Joshua Padgett
• 金额: $ 19.98万
• 依托单位: University of Arkansas
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-15 至 2023-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2206675&HistoricalAwards=false
• 关键词: Expressivity; Structure; Preserving; Deep; Neural

Partial differential equations (PDEs) arise naturally in the modeling and study of many natural, industrial, and financial phenomena. While there exist numerous techniques for approximating solutions to many types of PDEs, it is the case that certain problems of practical interest exhibit various types of pathology, which render many standard techniques either highly inefficient or unusable. Such situations often arise in the setting of nonlinear problems which are posed in high dimensions or exhibit singularities, such as in the study of solid-fuel combustion optimization, oil pipeline corrosion predictions, and high-frequency financial trading. This project intends to provide means of circumventing the aforementioned issues through the rigorous study and development of so-called structure-preserving deep neural networks (DNNs). While it has been experimentally observed that DNNs provide highly capable methods of approximating solutions to a large class of problems, it is the case that the theoretical justification of such observations is still in its early stage. To that end, this project will provide a much-needed theory for certain classes of high-dimensional nonlinear PDEs via a two-pronged approach: namely, explicit randomized methods will be constructed which demonstrate desired properties while simultaneously developing theoretical tools representing and studying a large class of objects via DNNs. This approach requires the use of numerous tools from applied mathematics, functional analysis, stochastic analysis, and novel DNN computations. This unique intersection of techniques will serve as the basis for the project's educational and training components, which aim to increase the presence of women, minorities, and other underrepresented groups in mathematical research. This goal will be accomplished through the training and mentoring first-generation and underrepresented students at both graduate and undergraduate levels.This project aims to address the question of whether or not it can be rigorously proven that there exist DNNs to approximate solutions to a large class of high-dimensional PDEs without suffering from the curse of dimensionality (CoD). The demonstration that DNNs are able to represent solutions to certain classes of high-dimensional nonlinear PDEs with a prescribed accuracy while not suffering from the CoD will fill a gap in the existing theory regarding machine learning algorithms. While the current focus is on studying the expressibility of DNNs with regard to solutions of PDEs, it is the case that this work will also serve as the foundation for extending such studies to other types of problems. This project will extend the existing theory of multilevel Picard (MLP) approximation methods to more general high-dimensional nonlinear PDEs, focusing on preserving inherent qualitative structures. The study of MLP approximations will also result in novel theoretical results regarding various stochastic fixed-point equations. Finally, the proposed work will provide explicit details on how to construct CoD-free DNN representations of various mathematical objects while also exploring theoretical issues related to popular activation functions. These ideas will be utilized in proving DNN-representation results and will provide a deeper understanding of how activation functions affect optimality.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

偏微分方程（PDE）在许多自然，工业和金融现象的建模和研究中自然出现。虽然存在许多技术来近似许多类型的偏微分方程的解决方案，但实际感兴趣的某些问题表现出各种类型的病理学，这使得许多标准技术要么效率很低，要么无法使用。这种情况经常出现在高维或奇异的非线性问题的设置中，例如在固体燃料燃烧优化，石油管道腐蚀预测和高频金融交易的研究中。该项目旨在通过严格研究和开发所谓的结构保留深度神经网络（DNN）来提供规避上述问题的方法。虽然已经通过实验观察到DNN为一大类问题提供了非常有能力的近似解决方案，但这种观察的理论论证仍处于早期阶段。为此，该项目将通过双管齐下的方法为某些类别的高维非线性偏微分方程提供急需的理论：即，将构建显式随机化方法，这些方法将展示所需的属性，同时开发通过DNN表示和研究一大类对象的理论工具。这种方法需要使用来自应用数学、泛函分析、随机分析和新型DNN计算的众多工具。这种独特的技术交叉将作为该项目教育和培训部分的基础，旨在增加妇女、少数民族和其他代表性不足的群体在数学研究中的存在。这一目标将通过在研究生和本科阶段培训和指导第一代和代表性不足的学生来实现。该项目旨在解决是否可以严格证明存在DNN来近似解决一大类高维偏微分方程而不会遭受维数灾难（CoD）的问题。DNN能够以规定的精度表示某些类别的高维非线性偏微分方程的解，同时不受CoD的影响，这一证明将填补现有机器学习算法理论的空白。虽然目前的重点是研究DNN在PDE解决方案方面的可表达性，但这项工作也将作为将此类研究扩展到其他类型问题的基础。本计画将现有的多层Picard（MLP）近似方法理论延伸至更一般的高维非线性偏微分方程，著重于保留固有的定性结构。对MLP近似的研究也将导致关于各种随机不动点方程的新的理论结果。最后，拟议的工作将提供关于如何构建各种数学对象的无编码DNN表示的明确细节，同时还将探索与流行激活函数相关的理论问题。这些想法将用于证明DNN表示结果，并将提供对激活函数如何影响最优性的更深入理解。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。