Comparative Study of Finite Element and Neural Network Discretizations for Partial Differential Equations

本研究通过对有限元法和有限神经元法的比较研究，将数据科学中的机器学习和科学与工程计算中的数值偏微分方程这两个不同的领域联系起来。有限元方法经过了数学家、科学家和工程师几十年的研究，在许多领域有丰富的数学理论。它们广泛用于科学计算和建模，以生成各种物理过程的精确模拟，最显着的是材料变形和流体力学。相比之下，深度神经网络相对较新，在过去十年中才得到广泛应用。在这么短的时间内，他们在各种各样的机器学习任务上表现出了卓越的经验表现，尤其是在计算机视觉和自然语言处理方面。尽管在经验上取得了巨大的成功，但对于深度神经网络为何以及如何运作得如此之好，人们的数学理解仍然非常有限。我们希望利用深度学习的成功来改进偏微分方程的数值方法，并利用对有限元方法的理论理解来更好地理解深度学习。该研究的跨学科性质也将为初级研究人员提供良好的培训经验。该项目将在为期三年的项目中每年资助1名研究生。分段多项式是逼近理论中最重要的泛函类之一。在偏微分方程的经典逼近理论和数值方法中，这些泛函类通常由与先验给定网格相关的线性泛函空间表示，例如样条和有限元空间。在深度学习中，函数类通常由一系列线性函数和坐标非线性的组合来表示。一个重要的非线性是整流线性单元（ReLU）函数及其幂函数（ReLUk）。由此产生的函数类，ReLUk-DNN，不形成线性向量空间，而是由一组高维参数非线性参数化。这个函数类可以用来求解偏微分方程，我们把得到的数值算法称为有限神经元法（FNM）。提出的研究课题包括：有限神经元方法的误差估计，任意高阶偏微分方程的一致性有限元的普遍构造，有限神经元方法如何以及为什么比相应的有限元方法给出更好的渐近误差估计，以及开发和分析使用有限神经元方法的有效算法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research connects two different fields, machine learning from data science and numerical partial differential equations from scientific and engineering computing, through the comparative study of the finite element method and finite neuron method. Finite element methods have undergone decades of study by mathematicians, scientists and engineers in many fields and there is a rich mathematical theory concerning them. They are widely used in scientific computing and modelling to generate accurate simulations of a wide variety of physical processes, most notably the deformation of materials and fluid mechanics. By contrast, deep neural networks are relatively new and have only been widely used in the last decade. In this short time, they have demonstrated remarkable empirical performance on a wide variety of machine learning tasks, most notably in computer vision and natural language processing. Despite this great empirical success, there is still a very limited mathematical understanding of why and how deep neural networks work so well. We hope to leverage the success of deep learning to improve numerical methods for partial differential equations and to leverage the theoretical understanding of the finite element method to better understand deep learning. The interdisciplinary nature of the research will also provide a good training experience for junior researchers. This project will support 1 graduate student each year of the three year project. Piecewise polynomials represent one of the most important functional classes in approximation theory. In classical approximation theory and numerical methods for partial differential equations, these functional classes are often represented by linear functional spaces associated with a priori given grids, for example, by splines and finite element spaces. In deep learning, function classes are typically represented by a composition of a sequence of linear functions and coordinate-wise non-linearities. One important non-linearity is the rectified linear unit (ReLU) function and its powers (ReLUk). The resulting functional class, ReLUk-DNN, does not form a linear vector space but is rather parameterized non-linearly by a high-dimensional set of parameters. This function class can be used to solve partial differential equations and we call the resulting numerical algorithms the finite neuron method (FNM). Proposed research topics include: error estimates for the finite neuron method, universal construction of conforming finite elements for arbitrarily high order partial differential equations, an investigation into how and why the finite neuron method gives a much better asymptotic error estimate than the corresponding finite element method, and the development and analysis of efficient algorithms for using the finite neuron method.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.