An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization
An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization
复制标题
DOI：
10.1007/s10107-019-01381-4
复制
发表时间：
2020-05-01
影响因子：
2.7
通讯作者：
Sra, Suvrit
中科院分区：
文献类型：
--
作者：
Hosseini, Reshad;Sra, Suvrit
我们考虑高斯混合模型（Gmm s）的最大似然估计。这个任务几乎总是通过期望最大化（EM）算法来解决（在理论和实践中）。新兴市场的成功得益于多种因素，其中以封闭形式实现正确定性约束的能力至关重要。我们提出了一种基于正定矩阵黎曼几何的EM替代方案，使用它我们将Gmm参数估计作为黎曼优化问题。令人惊讶的是，这样一个开箱即用的黎曼公式完全失败了，并且被证明比EM差得多。这促使我们更仔细地研究问题几何，并推导出一个更适合黎曼优化的更好的公式。然后，我们开发了优于EM的黎曼批处理和随机梯度算法，通常是显著的。我们提供了随机方法的非渐近收敛分析，这也是第一次（据我们所知）黎曼随机梯度的非渐近收敛分析。包括大量的实证结果，以证明我们的方法的有效性。
We consider maximum likelihood estimation for Gaussian Mixture Models (Gmm s). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM grounded in the Riemannian geometry of positive definite matrices, using which we cast Gmm parameter estimation as a Riemannian optimization problem. Surprisingly, such an out-of-the-box Riemannian formulation completely fails and proves much inferior to EM. This motivates us to take a closer look at the problem geometry, and derive a better formulation that is much more amenable to Riemannian optimization. We then develop Riemannian batch and stochastic gradient algorithms that outperform EM, often substantially. We provide a non-asymptotic convergence analysis for our stochastic method, which is also the first (to our knowledge) such global analysis for Riemannian stochastic gradient. Numerous empirical results are included to demonstrate the effectiveness of our methods.