Comparative Study of Finite Element and Neural Network Discretizations for Partial Differential Equations

偏微分方程有限元与神经网络离散化的比较研究

• 批准号: 2111387
• 负责人: Jonathan Siegel
• 金额: $ 55万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-15 至 2024-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2111387&HistoricalAwards=false
• 关键词: Comparative; Study; Finite; Element; Neural

This research connects two different fields, machine learning from data science and numerical partial differential equations from scientific and engineering computing, through the comparative study of the finite element method and finite neuron method. Finite element methods have undergone decades of study by mathematicians, scientists and engineers in many fields and there is a rich mathematical theory concerning them. They are widely used in scientific computing and modelling to generate accurate simulations of a wide variety of physical processes, most notably the deformation of materials and fluid mechanics. By contrast, deep neural networks are relatively new and have only been widely used in the last decade. In this short time, they have demonstrated remarkable empirical performance on a wide variety of machine learning tasks, most notably in computer vision and natural language processing. Despite this great empirical success, there is still a very limited mathematical understanding of why and how deep neural networks work so well. We hope to leverage the success of deep learning to improve numerical methods for partial differential equations and to leverage the theoretical understanding of the finite element method to better understand deep learning. The interdisciplinary nature of the research will also provide a good training experience for junior researchers. This project will support 1 graduate student each year of the three year project. Piecewise polynomials represent one of the most important functional classes in approximation theory. In classical approximation theory and numerical methods for partial differential equations, these functional classes are often represented by linear functional spaces associated with a priori given grids, for example, by splines and finite element spaces. In deep learning, function classes are typically represented by a composition of a sequence of linear functions and coordinate-wise non-linearities. One important non-linearity is the rectified linear unit (ReLU) function and its powers (ReLUk). The resulting functional class, ReLUk-DNN, does not form a linear vector space but is rather parameterized non-linearly by a high-dimensional set of parameters. This function class can be used to solve partial differential equations and we call the resulting numerical algorithms the finite neuron method (FNM). Proposed research topics include: error estimates for the finite neuron method, universal construction of conforming finite elements for arbitrarily high order partial differential equations, an investigation into how and why the finite neuron method gives a much better asymptotic error estimate than the corresponding finite element method, and the development and analysis of efficient algorithms for using the finite neuron method.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

本研究通过有限元方法和有限神经元方法的比较研究，将数据科学中的机器学习和科学与工程计算中的数值偏微分方程这两个不同的领域联系起来。有限元法已经被许多领域的数学家、科学家和工程师研究了几十年，并形成了丰富的数学理论。它们广泛用于科学计算和建模，以生成各种物理过程的精确模拟，最显着的是材料和流体力学的变形。相比之下，深度神经网络相对较新，在过去十年中才得到广泛应用。在这么短的时间内，他们在各种机器学习任务上表现出了卓越的经验表现，尤其是在计算机视觉和自然语言处理方面。尽管在经验上取得了巨大的成功，但对于深度神经网络为什么以及如何工作得这么好，仍然存在非常有限的数学理解。我们希望利用深度学习的成功来改进偏微分方程的数值方法，并利用对有限元方法的理论理解来更好地理解深度学习。研究的跨学科性质也将为初级研究人员提供良好的培训经验。该项目将支持一个研究生每年的三年期项目。分段多项式是逼近论中最重要的函数类之一。 在偏微分方程的经典逼近理论和数值方法中，这些函数类通常由与先验给定网格相关联的线性函数空间表示，例如样条和有限元空间。在深度学习中，函数类通常由一系列线性函数和坐标非线性的组合表示。一个重要的非线性是整流线性单元（ReLU）函数及其幂（ReLUk）。 由此产生的函数类ReLUk-DNN并不形成线性向量空间，而是通过一组高维参数进行非线性参数化。 这个函数类可以用来解决偏微分方程，我们称之为有限神经元方法（FNM）的数值算法。建议的研究课题包括：有限神经元方法的误差估计，任意高阶偏微分方程协调有限元的通用构造，有限神经元方法如何以及为什么比相应的有限元方法给出更好的渐近误差估计的研究，以及使用有限神经元方法的有效算法的开发和分析。该奖项反映了NSF的法定使命，通过使用基金会的知识价值和更广泛的影响审查标准进行评估，

项目成果

Greedy training algorithms for neural networks and applications to PDEs

• DOI: 10.1016/j.jcp.2023.112084
• 发表时间: 2021-07
• 期刊: J. Comput. Phys.
• 作者: Jonathan W. Siegel;Q. Hong;Xianlin Jin;Wenrui Hao;Jinchao Xu

Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

• DOI: 10.1007/s00365-023-09626-4
• 发表时间: 2021-06
• 期刊: Constructive Approximation
• 影响因子: 2.7
• 作者: Jonathan W. Siegel;Jinchao Xu

Sharp Bounds on the Approximation Rates, Metric Entropy, and n-Widths of Shallow Neural Networks

• DOI: 10.1007/s10208-022-09595-3
• 发表时间: 2021-01
• 期刊: Foundations of Computational Mathematics
• 影响因子: 3
• 作者: Jonathan W. Siegel;Jinchao Xu

Approximation properties of deep ReLU CNNs

• DOI: 10.1007/s40687-022-00336-0
• 发表时间: 2021-09
• 期刊: Research in the Mathematical Sciences
• 影响因子: 1.2
• 作者: Juncai He;Lin Li;Jinchao Xu

Optimal Convergence Rates for the Orthogonal Greedy Algorithm

正交贪婪算法的最优收敛率

• DOI: 10.1109/tit.2022.3147984
• 发表时间: 2022
• 期刊: IEEE Transactions on Information Theory
• 影响因子: 2.5
• 作者: Siegel, Jonathan W.;Xu, Jinchao
• 通讯作者: Xu, Jinchao