Nonconvex Optimization via MM Algorithms: Convergence Theory
Nonconvex Optimization via MM Algorithms: Convergence Theory
复制标题
通过 MM 算法进行非凸优化：收敛理论
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Hua Zhou
中科院分区：
文献类型：
--
作者：
K. Lange;Joong;Alfonso Landeros;Hua Zhou
优化最小化（MM）原则是一个非常普遍的框架，用于推导优化算法。它包括期望最大化（EM）算法，邻近梯度算法，凹凸过程，二次下界算法，和邻近距离算法作为特殊情况。除了在统计、优化和成像中的众多应用外，MM原理在大规模机器学习问题中也有广泛的应用，如矩阵完备、判别分析和非负矩阵分解。MM算法在求解非凸优化问题时，具有目标函数凸化、变量分离、数值稳定和易于实现等优点。然而，与其他优化算法的大量文献相比，MM算法的收敛性分析是分散的和特定的问题。本文给出了MM算法收敛性的统一处理方法。考虑到现代应用，结果包括非光滑的目标函数和非渐近分析。
The majorization-minimization (MM) principle is an extremely general framework for deriving optimization algorithms. It includes the expectation-maximization (EM) algorithm, proximal gradient algorithm, concave-convex procedure, quadratic lower bound algorithm, and proximal distance algorithm as special cases. Besides numerous applications in statistics, optimization, and imaging, the MM principle finds wide applications in large scale machine learning problems such as matrix completion, discriminant analysis, and nonnegative matrix factorizations. When applied to nonconvex optimization problems, MM algorithms enjoy the advantages of convexifying the objective function, separating variables, numerical stability, and ease of implementation. However, compared to the large body of literature on other optimization algorithms, the convergence analysis of MM algorithms is scattered and problem specific. This survey presents a unified treatment of the convergence of MM algorithms. With modern applications in mind, the results encompass non-smooth objective functions and nonasymptotic analysis.
DOI：
10.1137/20m1363388
发表时间：
2021
期刊：
SIAM journal on matrix analysis and applications : a publication of the Society for Industrial and Applied Mathematics
影响因子：
--
作者：
Won JH;Zhou H;Lange K
通讯作者：
Lange K
影响因子：
2.7
作者：
Bien, Jacob;Tibshirani, Robert J.
通讯作者：
Tibshirani, Robert J.
影响因子：
4.5
作者：
Hunter, DR;Li, RZ
通讯作者：
Li, RZ