Random Matrix Approaches to Approximate Bayesian Inference in Machine Learning

机器学习中近似贝叶斯推理的随机矩阵方法

• 批准号: 407712271
• 负责人: Professor Dr. Manfred Opper
• 金额: --
• 依托单位: Fachgebiet Methoden der Künstlichen Intelligenz (KI)
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2018
• 资助国家: 德国
• 起止时间: 2017-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/407712271?language=en
• 关键词: Random; Matrix; Approaches; Approximate; Bayesian

The Bayesian paradigm provides important methods for learning from data. It combines a probabilistic model for the generation of data together with prior knowledge over likely parameters within a probability distribution over the parameters of the model. However, practical applications of this idea to models with a large number of parameters are often plagued by computational problems related to the intractability of high--dimensional probability distributions. Approximate inference methods of machine learning provide algorithms for approximating such distributions by simpler ones - typically by multivariate Gaussian distributions. These inference methods yield often excellent results in applications. But, the update of covariance matrices (which give important information on uncertainties and dependencies between variables) of these Gaussian distributions within the iterative inference algorithms requires matrix operations per iteration which grows cubic in the number of parameters of the model. This makes the applications of such methods problematic when the number of variables is large. Hence, further approximations are necessary. And these may deteriorate the quality of the predictions. A second relevant problem is the fact that there is no guarantee of convergence for some popular inference algorithms. It is unclear if the failure to converge is an artefact of the algorithm or is related to the complexity of the Bayesian model. Motivated by recent research in the fields of information theory and statistical physics, this project will address these problems from a new angle. Assuming that data matrices can be considered as random (in a mathematically well-defined way), results of random matrix theory suggest novel ways to efficiently approximate the required matrix operations. These approximations are expected to perform well in the asymptotic limit when matrices are large. Random matrix methods will also provide new ways for analyzing the performance of iterative inference algorithms for large problems under certain statistical assumptions on the data. We will use these random matrix techniques to speed up existing algorithms as well as designing novel algorithms with optimized convergence properties. We will investigate the quality and robustness of such methods. Finally, we will validate our approach on various Bayesian models in machine learning and compare the performance with that of competing methods on simulated as well as real data.

贝叶斯范式提供了从数据中学习的重要方法。它将用于生成数据的概率模型与关于模型参数的概率分布内的可能参数的先验知识相结合。 然而，这一想法的实际应用中的模型与大量的参数往往困扰着计算问题的棘手性的高维概率分布。机器学习的近似推理方法提供了通过更简单的分布来近似这种分布的算法-通常是通过多变量高斯分布。这些推理方法在实际应用中往往能得到很好的结果。但是，迭代推理算法中这些高斯分布的协方差矩阵（其给出关于变量之间的不确定性和依赖性的重要信息）的更新需要每次迭代的矩阵运算，其在模型的参数的数量上立方地增长。这使得当变量的数量很大时，这些方法的应用成为问题。因此，进一步的近似是必要的。这可能会降低预测的质量。第二个相关的问题是一些流行的推理算法不能保证收敛。 目前还不清楚收敛失败是算法的人为因素还是与贝叶斯模型的复杂性有关。受信息论和统计物理领域最近研究的启发，本项目将从一个新的角度解决这些问题。假设数据矩阵可以被认为是随机的（在数学上定义良好的方式），随机矩阵理论的结果提出了新的方法来有效地近似所需的矩阵运算。当矩阵很大时，这些近似预计在渐近极限中表现良好。随机矩阵方法也将提供新的方法来分析迭代推理算法在某些统计假设下的性能。我们将使用这些随机矩阵技术，以加快现有的算法，以及设计新的算法优化收敛性能。我们将研究这些方法的质量和鲁棒性。最后，我们将在机器学习中的各种贝叶斯模型上验证我们的方法，并在模拟和真实的数据上与竞争方法的性能进行比较。