Neural Galerkin schemes with active learning for high-dimensional evolution equations
Neural Galerkin schemes with active learning for high-dimensional evolution equations
复制标题
DOI：
10.1016/j.jcp.2023.112588
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Joan Bruna;B. Peherstorfer;E. Vanden-Eijnden
中科院分区：
文献类型：
--
作者：
Joan Bruna;B. Peherstorfer;E. Vanden-Eijnden
深度神经网络已被证明可以在高维上提供精确的函数近似。然而，拟合网络参数需要翔实的训练数据，这在科学和工程应用中往往很难收集到。这项工作提出了基于深度学习的神经伽辽金方案，该方案通过主动学习生成用于数值求解高维偏微分方程的训练数据。神经Galerkin方案建立在Dirac-Frenkel变分原理的基础上，通过随时间顺序最小化残差来训练网络，这使得在偏微分方程描述的动力学指导下，以自适应的方式收集新的训练数据成为可能。这与其他机器学习方法形成对比，这些方法旨在及时拟合全局网络参数，而不考虑训练数据的获取。我们的发现是，主动收集所提出的神经伽辽金方案的训练数据的形式是在数字上实现高维网络表达能力的关键。数值实验表明，神经伽辽金方案有潜力模拟具有许多变量的现象和过程，这是传统和其他基于深度学习的求解器无法实现的，特别是当解决方案的特征局部演变时，例如在高维波传播问题和由Fokker-Planck和动力学方程描述的相互作用粒子系统中。
Deep neural networks have been shown to provide accurate function approximations in high dimensions. However, fitting network parameters requires informative training data that are often challenging to collect in science and engineering applications. This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time, which enables adaptively collecting new training data in a self-informed manner that is guided by the dynamics described by the partial differential equations. This is in contrast to other machine learning methods that aim to fit network parameters globally in time without taking into account training data acquisition. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions. Numerical experiments demonstrate that Neural Galerkin schemes have the potential to enable simulating phenomena and processes with many variables for which traditional and other deep-learning-based solvers fail, especially when features of the solutions evolve locally such as in high-dimensional wave propagation problems and interacting particle systems described by Fokker-Planck and kinetic equations.