Preconditioned Accelerated Gradient Descent Methods for Locally Lipschitz Smooth Objectives with Applications to the Solution of Nonlinear PDEs
Preconditioned Accelerated Gradient Descent Methods for Locally Lipschitz Smooth Objectives with Applications to the Solution of Nonlinear PDEs
复制标题
DOI：
10.1007/s10915-021-01615-8
复制
发表时间：
2020-06
影响因子：
2.5
通讯作者：
Jea-Hyun Park;A. Salgado;S. Wise
中科院分区：
文献类型：
--
作者：
Jea-Hyun Park;A. Salgado;S. Wise
我们为应用 Nesterov 的加速梯度下降法 (AGD) 来近似各类偏微分方程 (PDE) 的解奠定了理论基础。这是通过证明当其预处理版本 (PAGD) 用于最小化局部 Lipschitz 平滑、强凸目标函数时存在不变集和指数收敛率来实现的。我们引入了一个内置预处理器的二阶常微分方程 (ODE)，并表明 PAGD 是该 ODE 的显式时间离散化，它需要自然的时间步长限制来保证能量稳定性。在连续时间水平上，我们使用简单的能量参数展示了 ODE 解到其稳态的指数收敛。在离散层面，假设上述步长限制，证明了不变集的存在性，并通过模拟连续层面的能量论证和收敛，推导出PAGD方案的匹配指数收敛率。 PAGD 方法在数值偏微分方程中的应用通过使用伪谱方法进行空间离散化的某些非线性椭圆偏微分方程进行了演示，并进行了一些数值实验。结果证实了 PAGD 方法的全局几何和网格尺寸无关的收敛性，其加速速率比预条件梯度下降 (PGD) 方法有所提高。
We develop a theoretical foundation for the application of Nesterov’s accelerated gradient descent method (AGD) to the approximation of solutions of a wide class of partial differential equations (PDEs). This is achieved by proving the existence of aninvariant setand exponential convergence rates when its preconditioned version (PAGD) is applied to minimizelocally Lipschitzsmooth, strongly convex objective functionals. We introduce a second-order ordinary differential equation (ODE) with a preconditioner built-in and show that PAGD is an explicit time-discretization of this ODE, which requires a natural time step restriction for energy stability. At the continuous time level, we show an exponential convergence of the ODE solution to its steady state using a simple energy argument. At the discrete level, assuming the aforementioned step size restriction, the existence of an invariant set is proved and a matching exponential rate of convergence of the PAGD scheme is derived by mimicking the energy argument and the convergence at the continuous level. Applications of the PAGD method to numerical PDEs are demonstrated with certain nonlinear elliptic PDEs using pseudo-spectral methods for spatial discretization, and several numerical experiments are conducted. The results confirm the global geometric and mesh size-independent convergence of the PAGD method, with an accelerated rate that is improved over the preconditioned gradient descent (PGD) method.