An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization
An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization
复制标题
DOI：
10.1007/s11222-022-10120-3
复制
发表时间：
2022-07
影响因子：
2.2
通讯作者：
Wei Deng;Guang Lin;F. Liang
中科院分区：
文献类型：
--
作者：
Wei Deng;Guang Lin;F. Liang
我们提出了一种自适应加权随机梯度朗之万动力学（AWSGLD）算法，用于大数据问题的贝叶斯学习。该算法具有可扩展性，并具有自我调整机制：在模拟过程中自适应地压平高能区域并突出低能区域，从而在单次运行中极大地促进蒙特卡罗模拟和全局优化任务。自调整机制使得所提出的算法基本上不受局部陷阱的影响。理论上，通过证明平均场系统的稳定性并验证泊松方程解的存在性和规律性，我们建立了AWSGLD算法的收敛性，包括自适应参数的收敛性和加权平均估计器的收敛性。根据经验，AWSGLD 算法在多个基准数据集（包括 CIFAR100 和 SVHN）上进行了测试，用于优化和不确定性估计任务。数值结果表明其在现代机器学习任务的蒙特卡罗模拟和全局优化方面具有巨大潜力。
We propose an adaptively weighted stochastic gradient Langevin dynamics (AWSGLD) algorithm for Bayesian learning of big data problems. The proposed algorithm is scalable and possesses a self-adjusting mechanism: It adaptively flattens the high-energy region and protrudes the low-energy region during simulations such that both Monte Carlo simulation and global optimization tasks can be greatly facilitated in a single run. The self-adjusting mechanism enables the proposed algorithm to be essentially immune to local traps. Theoretically, by showing the stability of the mean-field system and verifying the existence and regularity properties of the solution of Poisson equation, we establish the convergence of the AWSGLD algorithm, including both the convergence of the self-adapting parameters and the convergence of the weighted averaging estimators. Empirically, the AWSGLD algorithm is tested on multiple benchmark datasets including CIFAR100 and SVHN for both optimization and uncertainty estimation tasks. The numerical results indicate its great potential in Monte Carlo simulation and global optimization for modern machine learning tasks.