Exponentiated) Stochastic Gradient Descent for L1 Constrained Problems
Exponentiated) Stochastic Gradient Descent for L1 Constrained Problems
复制标题
L1 约束问题的指数）随机梯度下降
DOI：
--
复制
发表时间：
2008
期刊：
影响因子：
--
通讯作者：
Ambuj Tewari
中科院分区：
文献类型：
--
作者：
S. Kakade;Ambuj Tewari
该注释是Sham Kakade，Dean Foster和Eye dev-dar。该说明表明，当用作一般凸出损失函数的优化工具时对于有监督的学习问题，我们希望在许多无关的特征的情况下，尤其是在轻度假设下的维度数量。有效的 - 具有样本复杂性，仅在特征总数和计算复杂性中仅是对数，该计算复杂性仅在特征总数（忽略对数因子）中是线性。
This note is by Sham Kakade, Dean Foster, and Eyal Even-Dar. It is intended as an introductory piece on solving L1 constrained problems with online methods. Convex optimization problems with L1 constraints frequently underly solving such tasks as feature selection problems and obtaining sparse representations. This note shows that the exponentiated gradient algorithm (of Kivinen and Warmuth (1997)) when used as a stochastic gradient descent algorithm is quite effective as an optimization tool under general convex loss functions — requiring a number of gradient steps that is logarithmic in the number of dimensions under mild assumptions. In particular, for supervised learning problems in which we desire to approximately minimize some general convex loss (including the square, logistic, hinge, or absolute loss) in the presence of many irrelevant features, this algorithm is efficient — with a sample complexity that is only logarithmic in the total number of features and a computational complexity that is only linear in the total number of features (ignoring log factors).