Choose Your Path Wisely: Gradient Descent in a Bregman Distance Framework
Choose Your Path Wisely: Gradient Descent in a Bregman Distance Framework
复制标题
DOI：
10.1137/20m1357500
复制
发表时间：
2017-12
期刊：
影响因子：
--
通讯作者：
Martin Benning;M. Betcke;Matthias Joachim Ehrhardt;C. Schonlieb
中科院分区：
文献类型：
--
作者：
Martin Benning;M. Betcke;Matthias Joachim Ehrhardt;C. Schonlieb
我们提出将梯度下降的特殊形式（在文献中称为线性布雷格曼迭代）扩展到更大类别的非凸函数。我们将梯度下降设置中的经典（平方）二范数度量替换为基于适当的、凸的和下半连续函数的广义布雷格曼距离。所提出的算法是众多众所周知的优化方法的概括。对于满足 Kurdyka-\L ojasiewicz 性质的函数，其全局收敛性得到了证明。示例表明，对于 Bregman 距离的适当选择，与传统的梯度下降相比，该方法允许沿着常规解路径进行迭代。线性化 Bregman 迭代与早期停止相结合的有效性在并行磁共振成像、盲反卷积以及图像分类的应用中得到了证明。
We propose an extension of a special form of gradient descent --- in the literature known as linearised Bregman iteration --- to a larger class of non-convex functionals. We replace the classical (squared) two norm metric in the gradient descent setting with a generalised Bregman distance, based on a proper, convex and lower semi-continuous functional. The proposed algorithm is a generalisation of numerous well-known optimisation methods. Its global convergence is proven for functions that satisfy the Kurdyka-\L ojasiewicz property. Examples illustrate that for suitable choices of Bregman distances this method --- in contrast to traditional gradient descent --- allows iterating along regular solution-paths. The effectiveness of the linearised Bregman iteration in combination with early stopping is illustrated for the applications of parallel magnetic resonance imaging, blind deconvolution as well as image classification.