Greedy training algorithms for neural networks and applications to PDEs
Greedy training algorithms for neural networks and applications to PDEs
复制标题
DOI：
10.1016/j.jcp.2023.112084
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Jonathan W. Siegel;Q. Hong;Xianlin Jin;Wenrui Hao;Jinchao Xu
中科院分区：
文献类型：
--
作者：
Jonathan W. Siegel;Q. Hong;Xianlin Jin;Wenrui Hao;Jinchao Xu
最近，神经网络已广泛应用于求解偏微分方程（PDE）。尽管这些方法已被证明在实际工程问题上非常成功，但无论从理论上还是从经验上来看，它们都没有被证明能够以任意高精度收敛到基础的 PDE 解。主要困难在于解决神经网络离散化带来的高度非凸优化问题，这些问题在理论上和实践上都难以处理。我们这项工作的目标是采取措施纠正这一问题。为此，我们为浅层神经网络开发了一种新颖的贪婪训练算法。我们的方法既适用于偏微分方程的变分公式，也适用于物理通知神经网络（PINN）开创的残差最小化公式。我们分析了该方法，并在从浅层网络定义的函数类求解偏微分方程时获得了先验误差界，随着网络规模的增加，该方法严格地建立了收敛性。最后，我们在几个基准示例（包括高维偏微分方程）上测试该算法，以确认理论收敛速度。尽管该方法相对于有限元方法等传统方法昂贵，但我们将这项工作视为基于神经网络的方法的概念证明，它表明基于神经网络的数值方法可以证明是严格收敛的。
Recently, neural networks have been widely applied for solving partial differential equations (PDEs). Although such methods have been proven remarkably successful on practical engineering problems, they have not been shown, theoretically or empirically, to converge to the underlying PDE solution with arbitrarily high accuracy. The primary difficulty lies in solving the highly non-convex optimization problems resulting from the neural network discretization, which are difficult to treat both theoretically and practically. It is our goal in this work to take a step toward remedying this. For this purpose, we develop a novel greedy training algorithm for shallow neural networks. Our method is applicable to both the variational formulation of the PDE and also to the residual minimization formulation pioneered by physics informed neural networks (PINNs). We analyze the method and obtain a priori error bounds when solving PDEs from the function class defined by shallow networks, which rigorously establishes the convergence of the method as the network size increases. Finally, we test the algorithm on several benchmark examples, including high dimensional PDEs, to confirm the theoretical convergence rate. Although the method is expensive relative to traditional approaches such as finite element methods, we view this work as a proof of concept for neural network-based methods, which shows that numerical methods based upon neural networks can be shown to rigorously converge.