PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence
PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence
复制标题
DOI：
10.1016/j.jcp.2023.112016
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Justin A. Sirignano;J. MacArt;K. Spiliopoulos
中科院分区：
文献类型：
--
作者：
Justin A. Sirignano;J. MacArt;K. Spiliopoulos
最近的研究使用深度学习来开发科学和工程中的偏微分方程（PDE）模型。偏微分方程的函数形式由神经网络确定，神经网络参数根据可用数据进行校准。嵌入式神经网络的校准可以通过在PDE上进行优化来执行。受这些应用的启发，我们严格研究了一类带神经网络项的线性椭圆偏微分方程的优化问题。PDE中的神经网络参数使用梯度下降进行优化，其中梯度使用伴随PDE进行评估。随着参数数量的增加，偏微分方程和伴随偏微分方程收敛到非局部偏微分方程系统。利用这个极限偏微分方程系统，我们能够证明神经网络-偏微分方程在优化过程中收敛到全局最小值。最后，我们使用这种伴随方法来训练神经网络模型在流体力学中的应用，其中的神经网络功能作为一个封闭的模型的雷诺平均Navier-Stokes（RANS）方程。RANS神经网络模型在多个湍流通道流数据集上进行训练，并在不同雷诺数下进行样本外评估。
Recent research has used deep learning to develop partial differential equation (PDE) models in science and engineering. The functional form of the PDE is determined by a neural network, and the neural network parameters are calibrated to available data. Calibration of the embedded neural network can be performed by optimizing over the PDE. Motivated by these applications, we rigorously study the optimization of a class of linear elliptic PDEs with neural network terms. The neural network parameters in the PDE are optimized using gradient descent, where the gradient is evaluated using an adjoint PDE. As the number of parameters become large, the PDE and adjoint PDE converge to a non-local PDE system. Using this limit PDE system, we are able to prove convergence of the neural network-PDE to a global minimum during the optimization. Finally, we use this adjoint method to train a neural network model for an application in fluid mechanics, in which the neural network functions as a closure model for the Reynolds-averaged Navier–Stokes (RANS) equations. The RANS neural network model is trained on several datasets for turbulent channel flow and is evaluated out-of-sample at different Reynolds numbers.