Characterizing Implicit Bias in Terms of Optimization Geometry
Characterizing Implicit Bias in Terms of Optimization Geometry
复制标题
DOI：
--
复制
发表时间：
2018-02
期刊：
影响因子：
--
通讯作者：
Suriya Gunasekar;Jason D. Lee;Daniel Soudry;N. Srebro
中科院分区：
文献类型：
--
作者：
Suriya Gunasekar;Jason D. Lee;Daniel Soudry;N. Srebro
我们研究了在优化欠定线性回归或可分线性分类问题时，通用优化方法（如镜像下降、自然梯度下降和最陡下降）对不同势能和范数的隐式偏差。我们探讨的问题，是否特定的全局最小值（在许多可能的全局最小值）达到的算法可以在潜在的或规范的优化几何特征，并独立于超参数的选择，如步长和动量。
We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or separable linear classification problems. We explore the question of whether the specific global minimum (among the many possible global minima) reached by an algorithm can be characterized in terms of the potential or norm of the optimization geometry, and independently of hyperparameter choices such as step-size and momentum.