Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models
Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models
复制标题
DOI：
10.48550/arxiv.2306.03249
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Alexander Lin;Bahareh Tolooshams;Yves Atchad'e;Demba E. Ba
中科院分区：
文献类型：
--
作者：
Alexander Lin;Bahareh Tolooshams;Yves Atchad'e;Demba E. Ba
潜高斯模型在统计学和机器学习方面有着丰富的历史，其应用范围从因子分析到压缩感知再到时间序列分析。最大化这些模型的可能性的经典方法是期望最大化（EM）算法。对于高维潜变量和大型数据集的问题，EM的可扩展性很差，因为它需要反转与数据点数量一样多的大型协方差矩阵。我们引入概率展开，一种方法，结合蒙特卡罗抽样与迭代线性求解器，以规避矩阵求逆。我们的理论分析表明，展开和反向传播通过迭代求解器可以加速梯度估计的最大似然估计。在模拟和真实的数据的实验中，我们证明了概率展开学习潜在的高斯模型比梯度EM快一个数量级，模型性能损失最小。
Latent Gaussian models have a rich history in statistics and machine learning, with applications ranging from factor analysis to compressed sensing to time series analysis. The classical method for maximizing the likelihood of these models is the expectation-maximization (EM) algorithm. For problems with high-dimensional latent variables and large datasets, EM scales poorly because it needs to invert as many large covariance matrices as the number of data points. We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversion. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.