Stochastic Subgradient Method Converges on Tame Functions
Stochastic Subgradient Method Converges on Tame Functions
复制标题
DOI：
10.1007/s10208-018-09409-5
复制
发表时间：
2020-02-01
影响因子：
3
通讯作者：
Lee, Jason D.
中科院分区：
文献类型：
--
作者：
Davis, Damek;Drusvyatskiy, Dmitriy;Lee, Jason D.
这项工作考虑了一个问题：在没有光滑度和凸度的情况下，随机亚级别方法可以保证什么融合可以保证？我们证明，在任何半gebraic局部Lipschitz函数上，随机亚级别方法都会产生一阶静止的限制点。更普遍地，我们的结果适用于具有惠特尼分层图的任何功能。特别是，这项工作赋予了随机亚级别的方法及其近端扩展，并具有严格的合并保证，可保证在数据科学中引起的广泛问题，包括所有流行的深度学习体系结构。
This work considers the question: what convergence guarantees does the stochastic subgradient method have in the absence of smoothness and convexity? We prove that the stochastic subgradient method, on any semialgebraic locally Lipschitz function, produces limit points that are all first-order stationary. More generally, our result applies to any function with a Whitney stratifiable graph. In particular, this work endows the stochastic subgradient method, and its proximal extension, with rigorous convergence guarantees for a wide class of problems arising in data science-including all popular deep learning architectures.