Efficient Monte Carlo Algorithms for Bayesian Inference

当前应用统计学和机器学习产生的数据集规模非常大，需要大型模型进行分析。贝叶斯推理和全局优化是从这类数据中学习的两种强大方法，但数据集的庞大规模和由此产生的计算困难极大地限制了这些方法的适用性。本项目的研究旨在提高这些方法的计算效率，从而大大扩展它们对大数据集分析的有用性。本研究的方法和算法将在现代分布式计算平台上实现，并免费提供给科学界。研究结果将在统计学和机器学习中有广泛的应用。具体来说，将研究在马尔可夫链蒙特卡罗（MCMC）中使用小批量。MCMC可能是贝叶斯统计推断中使用最广泛的计算方法。由于马尔可夫链模拟的每一步都需要扫描所有的观测值，对于一个大的数据集，这种计算是令人望而却步的。另一方面，在机器学习领域，研究人员发现随机优化技术一次只检查一小批数据点，可以提供出色的性能。在本项目中，将开发一个统一基于小批量的MCMC和全局优化的框架。结果表明，对后验分布的模拟可以通过仅依赖于小批量的具有metropolis - hastings更新的MCMC过程来近似。该方法将与等效能量采样相结合，实现统一的仿真和全局优化方法。这个框架将允许我们改进MCMC方法和非凸全局优化方法的性能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data sets arising from current applications of statistics and machine learning are of very large size and require large models for their analysis. Bayesian inference and global optimization are two powerful methods for learning from such data, but the large size of the data sets and the resulting computational difficulties greatly limit the applicability of these methods. The research in this project aims to increase computational efficiency of these methods, thereby substantially expanding their usefulness for the analysis of large data sets. The methods and algorithms from this research will be implemented on modern distributed computing platforms and made freely available for the scientific community. The results will have wide applications in statistics and machine learning.Specifically, the use of mini-batches in Markov Chain Monte Carlo (MCMC) will be investigated. MCMC is perhaps the most widely used computational approach for Bayesian statistical inference. Since each step in the simulation of the Markov chain requires the scanning of all the observations, for a large data set this computation is prohibitive. On the other hand, in the area of machine learning researchers have found that stochastic optimization techniques, which examine only a mini-batch of data points at a time, can deliver excellent performance. In this project, a framework for unifying mini-batch based MCMC and global optimization will be developed. It is showed that simulation from of a tempered version of the posterior distribution can be approximated by a MCMC process with Metropolis-Hasting updates that depend only on mini-batches. This approach will be combined with eqi-energy sampling to achieve a unified simulation and global optimization methodology. This framework will allow us to improve the performance of both MCMC methods and non-convex global optimization methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.