Stochastic Dual Coordinate Ascent with Alternating Direction Method of Multipliers
Stochastic Dual Coordinate Ascent with Alternating Direction Method of Multipliers
复制标题
DOI：
--
复制
发表时间：
2014-06
期刊：
影响因子：
--
通讯作者：
Taiji Suzuki
中科院分区：
文献类型：
--
作者：
Taiji Suzuki
我们提出了一种新的随机双坐标上升技术，可以应用于广泛的正则化学习问题。我们的方法是基于乘法器的交替方向法（ADMM）来处理复杂的正则化函数，如结构化正则化。虽然原始ADMM是批处理方法，但该方法提供了随机更新规则，每次迭代只需要一个或几个样本观测值。此外，我们的方法可以自然地进行小批量更新，并加快了收敛速度。我们证明，在温和的假设下，我们的方法是指数收敛的。数值实验表明，该方法是有效的。
We propose a new stochastic dual coordinate ascent technique that can be applied to a wide range of regularized learning problems. Our method is based on alternating direction method of multipliers (ADMM) to deal with complex regularization functions such as structured regularizations. Although the original ADMM is a batch method, the proposed method offers a stochastic update rule where each iteration requires only one or few sample observations. Moreover, our method can naturally afford mini-batch update and it gives speed up of convergence. We show that, under mild assumptions, our method converges exponentially. The numerical experiments show that our method actually performs efficiently.