EM vs MM: A Case Study.
EM vs MM: A Case Study.
复制标题
EM 与 MM：案例研究。
DOI：
10.1016/j.csda.2012.05.018
复制
发表时间：
2012
影响因子：
1.8
通讯作者：
Zhang,Yiwen
中科院分区：
文献类型：
--
作者：
Zhou,Hua;Zhang,Yiwen
期望最大化（EM）算法是统计学中应用最广泛的优化方法之一。近年来，人们已经认识到EM算法是更普遍的最小化最大化（MM）原理的一个特例。两种算法都在第一步（E或M）中创建代理函数，该函数在第二个M步中最大化。这两步过程总是使目标函数上坡并迭代直到参数收敛。这两种算法的不同之处在于代理函数的构造方式。EM算法的期望步骤依赖于计算条件期望，而MM算法的最小化步骤建立在巧妙使用不等式的基础上。对于许多问题，EM和MM的推导产生相同的算法。这篇说明性的笔记介绍了用于估计dirichlet -多项式分布参数的两种算法的构造。这种特殊情况是有趣的，因为EM和MM衍生导致两种不同的算法，具有完全不同的操作特性。EM算法收敛速度快，但在M步中需要解决一个非平凡的最大化问题。相比之下，MM的更新非常简单，但收敛速度很慢。导出了一种EM-MM混合算法，该算法在某些参数范围内收敛速度比MM算法快。从统一MM的角度对三种算法的局部收敛速度进行了理论研究，并通过数值算例进行了比较。
The celebrated expectation–maximization (EM) algorithm is one of the most widely used optimization methods in statistics. In recent years it has been realized that EM algorithm is a special case of the more general minorization–maximization (MM) principle. Both algorithms create a surrogate function in the first (E or M) step that is maximized in the second M step. This two step process always drives the objective function uphill and is iterated until the parameters converge. The two algorithms differ in the way the surrogate function is constructed. The expectation step of the EM algorithm relies on calculating conditional expectations, while the minorization step of the MM algorithm builds on crafty use of inequalities. For many problems, EM and MM derivations yield the same algorithm. This expository note walks through the construction of both algorithms for estimating the parameters of the Dirichlet-Multinomial distribution. This particular case is of interest because EM and MM derivations lead to two different algorithms with completely distinct operating characteristics. The EM algorithm converges quickly but involves solving a nontrivial maximization problem in the M step. In contrast the MM updates are extremely simple but converge slowly. An EM–MM hybrid algorithm is derived which shows faster convergence than the MM algorithm in certain parameter regimes. The local convergence rates of the three algorithms are studied theoretically from the unifying MM point of view and also compared on numerical examples.