On Riemannian Stochastic Approximation Schemes with Fixed Step-Size
On Riemannian Stochastic Approximation Schemes with Fixed Step-Size
复制标题
固定步长黎曼随机逼近方案
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
S. Said
中科院分区：
文献类型：
--
作者：
Alain Durmus;P. Jiménez;É. Moulines;S. Said
本文在黎曼框架下研究了固定步长随机逼近格式，包括随机梯度格式。它的动机是几个应用程序，测地线可以明确计算，他们的使用加速原油欧几里德方法。一个固定的步长计划定义了一个家庭的时间齐次马尔可夫链，参数化的步长。在这里，使用这个配方，非渐近性能界推导出，在李雅普诺夫条件下。然后，对任意步长，证明了相应的马尔可夫链存在唯一的平稳分布，且是几何遍历的。这一结果产生了一个家庭的平稳分布索引的步长，这是进一步表明收敛到狄拉克措施，集中在解决问题的手，作为步长为0。最后，建立了这种收敛的渐近速度，通过渐近展开的偏见，和一个中心极限定理。
This paper studies fixed step-size stochastic approximation (SA) schemes, including stochastic gradient schemes, in a Riemannian framework. It is motivated by several applications, where geodesics can be computed explicitly, and their use accelerates crude Euclidean methods. A fixed step-size scheme defines a family of time-homogeneous Markov chains, parametrized by the step-size. Here, using this formulation, non-asymptotic performance bounds are derived, under Lyapunov conditions. Then, for any step-size, the corresponding Markov chain is proved to admit a unique stationary distribution, and to be geometrically ergodic. This result gives rise to a family of stationary distributions indexed by the step-size, which is further shown to converge to a Dirac measure, concentrated at the solution of the problem at hand, as the step-size goes to 0. Finally, the asymptotic rate of this convergence is established, through an asymptotic expansion of the bias, and a central limit theorem.
影响因子：
2.7
作者：
Hosseini, Reshad;Sra, Suvrit
通讯作者：
Sra, Suvrit