Projection-Based Regularized Dual Averaging for Stochastic Optimization
Projection-Based Regularized Dual Averaging for Stochastic Optimization
复制标题
用于随机优化的基于投影的正则化对偶平均
DOI：
10.1109/tsp.2019.2908901
复制
发表时间：
2019
影响因子：
5.4
通讯作者：
Yukawa Masahiro
中科院分区：
文献类型：
--
作者：
Ushio Asahi;Yukawa Masahiro
提出了一种基于正则化对偶平均(RDA)方法的随机优化框架。该方法不同于以往的RDA研究，主要体现在三个方面。首先，利用“随机”闭凸集的平方距离损失函数来保证稳定性。其次，同时使用稀疏性提升度量(由某种比例型自适应滤波算法隐含地使用)和二次加权的ℓ1正则化。由于损失函数的光滑性，步长和正则化参数都是恒定的。这三个差异产生了良好的稀疏性、较高的估计精度和对正则化参数的选择不敏感。数值算例表明，与已有的方法(包括AdaGrad方法和自适应近邻向前向后分裂方法)相比，该方法在应用于真实/合成数据的回归和分类中具有显著的优势。
We propose a novel stochastic-optimization framework based on the regularized dual averaging (RDA) method. The proposed approach differs from the previous studies of RDA in three major aspects. First, the squared-distance loss function to a “random” closed convex set is employed for stability. Second, a sparsity-promoting metric (used implicitly by a certain proportionate-type adaptive filtering algorithm) and a quadratically-weighted ℓ1regularizer are used simultaneously. Third, the step size and regularization parameters are both constant due to the smoothness of the loss function. These three differences yield an excellent sparsity-seeking property, high estimation accuracy, and insensitivity to the choice of the regularization parameter. Numerical examples show the remarkable advantages of the proposed method over the existing methods (including AdaGrad and the adaptive proximal forward-backward splitting method) in applications to regression and classification with real/synthetic data.