Nonasymptotic Estimates for Stochastic Gradient Langevin Dynamics Under Local Conditions in Nonconvex Optimization
Nonasymptotic Estimates for Stochastic Gradient Langevin Dynamics Under Local Conditions in Nonconvex Optimization
复制标题
DOI：
10.1007/s00245-022-09932-6
复制
发表时间：
2019-10
影响因子：
1.8
通讯作者：
Ying Zhang;Ömer Deniz Akyildiz;T. Damoulas;S. Sabanis
中科院分区：
文献类型：
--
作者：
Ying Zhang;Ömer Deniz Akyildiz;T. Damoulas;S. Sabanis
在本文中，我们关注非凸优化中使用的抽样算法的非渐近分析。特别地，我们获得了一种称为随机梯度朗格万动力学（SGLD）的流行算法在Wasserstein-1和Wasserstein-2距离上的非渐近估计。此外，上述Wasserstein-2收敛结果可用于建立期望超额风险的非渐近误差界。重要的是，这些结果是在局部Lipschitz条件和局部耗散条件下得到的，我们消除了数据流中的均匀依赖。我们通过介绍变分推理和索引跟踪优化的例子来说明这种松弛的重要性。
In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the aforementioned Wasserstein-2 convergence result can be applied to establish a non-asymptotic error bound for the expected excess risk. Crucially, these results are obtained under a local Lipschitz condition and a local dissipativity condition where we remove the uniform dependence in the data stream. We illustrate the importance of this relaxation by presenting examples from variational inference and from index tracking optimization.