Stochastic Composite Convex Minimization with Affine Constraints
Stochastic Composite Convex Minimization with Affine Constraints
复制标题
DOI：
10.1109/acssc.2018.8645298
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
K. Slavakis
中科院分区：
文献类型：
--
作者：
K. Slavakis
本文介绍了一种新方法的基本要素，即随机 Fejér 单调混合最速下降法（S-FM-HSDM），旨在解决仿射约束和复合凸最小化任务。最小化任务是未知的；噪声污染了有关复合损失函数和仿射约束的信息。 S-FM-HSDM 生成随机变量序列，这些随机变量序列在某些条件下并相对于概率空间，逐点收敛到无噪声最小化任务的解。 S-FM-HSDM 具有最先进的随机逼近技术的理想属性，例如变量分割和恒定步长（学习率）。此外，它提供了一种通过适当映射的定点集来利用仿射约束信息的新颖方法。在 S-FM-HSDM 的后代中，分层递归最小二乘法 (HRLS) 利用了 S-FM-HSDM 对仿射约束的多功能性，并通过生成收敛于分层优化任务解的估计序列，为 LS 提供了一种新颖的转折：最小化集合最小二乘损失最小化集上的凸损失。对合成 $\ell_{1}$-范数正则化 LS 任务的数值测试表明，HRLS 与几种最先进的凸、非凸、随机逼近和在线学习对应任务相比具有优势。
This paper presents the basic ingredients of a novel method, the stochastic Fejér-monotone hybrid steepest descent method S-FM-HSDM), designed to solve affinely constrained and composite convex minimization tasks. The minimization task is not known exactly; noise contaminates the information about the composite loss function and the affine constraints. S-FM-HSDM generates sequences of random variables that, under certain conditions and with respect to a probability space, converge pointwise to solutions of the noiseless minimization task. S-FM-HSDM enjoys desirable attributes of state-of-the-art stochastic-approximation techniques such as splitting of variables and constant step size (learning rate). Furthermore, it provides a novel way of exploiting the information about the affine constraints via fixed-point sets of appropriate mappings. Among the offsprings of S-FM-HSDM, the hierarchical recursive least squares (HRLS) takes advantage of S-FM-HSDM’s versatility toward affine constraints and offers a novel twist to LS by generating sequences of estimates that converge to solutions of a hierarchical optimization task: Minimize a convex loss over the set of minimizers of the ensemble least-squares loss. Numerical tests on a synthetic $\ell_{1}$-norm regularized LS task show that HRLS compares favorably to several state-of-the-art convex, as well as non-convex, stochastic-approximation and online-learning counterparts.