Simple Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization
Simple Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization
复制标题
DOI：
--
复制
发表时间：
2019-01
期刊：
影响因子：
--
通讯作者：
Michael R. Metel;A. Takeda
中科院分区：
文献类型：
--
作者：
Michael R. Metel;A. Takeda
我们的工作着重于使用非平滑型非凸正规剂优化平滑的非凸丢失函数的随机梯度方法。对这类问题的研究非常有限，直到最近，还没有报道过非反应收敛结果。我们提出了两种简单的随机梯度算法，用于有限的和一般的随机优化问题，与当前的最新技术相比，它们具有优越的收敛复杂性。我们还比较了算法在实践中的性能，以最小化经验风险。
Our work focuses on stochastic gradient methods for optimizing a smooth non-convex loss function with a non-smooth non-convex regularizer. Research on this class of problem is quite limited, and until recently no non-asymptotic convergence results have been reported. We present two simple stochastic gradient algorithms, for finite-sum and general stochastic optimization problems, which have superior convergence complexities compared to the current state-of-the-art. We also compare our algorithms' performance in practice for empirical risk minimization.