Constrained Langevin Algorithms with L-mixing External Random Variables-猫眼课题宝

DOI：

10.48550/arxiv.2205.14192

发表时间：

2022-05

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Yu Zheng;Andrew G. Lamperski

朗之万算法是一种增加了加性噪声的梯度下降方法，广泛用于马尔可夫链蒙特卡罗（MCMC）采样，优化和机器学习。近年来，非凸学习的Langevin算法的非渐近分析得到了广泛的研究。对于在具有IID数据变量的紧致凸域上具有非凸损失的约束问题，投影Langevin算法在$1$-Wasserstein距离中实现了$O（T^{-1/4}（\log T）^{1/2}）$与其目标分布的偏差[27]。在本文中，我们得到了$O（T^{-1/2} \log T）$在$1$-Wasserstein距离的偏差为$L$-混合数据变量和多面体约束（不一定有界）的非凸损失。这改进了以前的约束问题的界限，并匹配最知名的无约束问题的界限。

Langevin algorithms are gradient descent methods augmented with additive noise, and are widely used in Markov Chain Monte Carlo (MCMC) sampling, optimization, and machine learning. In recent years, the non-asymptotic analysis of Langevin algorithms for non-convex learning has been extensively explored. For constrained problems with non-convex losses over a compact convex domain with IID data variables, the projected Langevin algorithm achieves a deviation of $O(T^{-1/4} (\log T)^{1/2})$ from its target distribution [27] in $1$-Wasserstein distance. In this paper, we obtain a deviation of $O(T^{-1/2} \log T)$ in $1$-Wasserstein distance for non-convex losses with $L$-mixing data variables and polyhedral constraints (which are not necessarily bounded). This improves on the previous bound for constrained problems and matches the best-known bound for unconstrained problems.

Constrained Langevin Algorithms with L-mixing External Random Variables

文献摘要

相似文献