ON THE BEHAVIOR OF THE EXPECTATION-MAXIMIZATION ALGORITHM FOR MIXTURE MODELS
ON THE BEHAVIOR OF THE EXPECTATION-MAXIMIZATION ALGORITHM FOR MIXTURE MODELS
复制标题
混合模型期望最大化算法的行为
DOI：
10.1109/globalsip.2018.8646506
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Meisam Razaviyayn
中科院分区：
文献类型：
--
作者：
Babak Barazandeh;Meisam Razaviyayn
有限混合模型是不同数据科学学科中最流行的统计模型之一。尽管它们具有广泛的适用性，但这些模型下的推理通常会导致计算上具有挑战性的非凸问题。虽然期望最大化（EM）算法是解决这些非凸问题的最流行的方法，但该算法的行为还没有得到很好的理解。在这项工作中，我们专注于混合拉普拉斯（或高斯）分布的情况下。我们首先分析两个一维拉普拉斯分布的简单等权混合，并表明群体最大似然估计问题的每个局部最优解都是全局最优的。然后，我们证明了EM算法收敛到地面真理参数几乎必然与随机初始化。我们的结果推广了高斯分布到拉普拉斯分布的已有结果。然后，我们数值研究了具有两个以上组分的混合模型的行为。基于我们大量的数值实验，我们提出了一种新的随机方法来估计混合模型中各分量的均值。我们的数值实验表明，我们的算法优于朴素EM算法在几乎所有的情况下。
Finite mixture models are among the most popular statistical models used in different data science disciplines. Despite their broad applicability, inference under these models typically leads to computationally challenging non-convex problems. While the Expectation-Maximization (EM) algorithm is the most popular approach for solving these non-convex problems, the behavior of this algorithm is not well understood. In this work, we focus on the case of mixture of Laplacian (or Gaussian) distribution. We start by analyzing a simple equally weighted mixture of two single dimensional Laplacian distributions and show that every local optimum of the population maximum likelihood estimation problem is globally optimal. Then, we prove that the EM algorithm converges to the ground truth parameters almost surely with random initialization. Our result extends the existing results for Gaussian distribution to Laplacian distribution. Then we numerically study the behavior of mixture models with more than two components. Motivated by our extensive numerical experiments, we propose a novel stochastic method for estimating the mean of components of a mixture model. Our numerical experiments show that our algorithm outperforms the Naïve EM algorithm in almost all scenarios.
DOI：
--
发表时间：
2016-09
期刊：
ArXiv
影响因子：
--
作者：
C. Daskalakis;Christos Tzamos;Manolis Zampetakis
通讯作者：
C. Daskalakis;Christos Tzamos;Manolis Zampetakis