Efficient Monte Carlo Algorithms for Bayesian Inference

用于贝叶斯推理的高效蒙特卡罗算法

• 批准号: 1811920
• 负责人: Wing Hung Wong
• 金额: $ 20万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1811920&HistoricalAwards=false
• 关键词: Efficient; Monte; Carlo; Algorithms; Bayesian

Data sets arising from current applications of statistics and machine learning are of very large size and require large models for their analysis. Bayesian inference and global optimization are two powerful methods for learning from such data, but the large size of the data sets and the resulting computational difficulties greatly limit the applicability of these methods. The research in this project aims to increase computational efficiency of these methods, thereby substantially expanding their usefulness for the analysis of large data sets. The methods and algorithms from this research will be implemented on modern distributed computing platforms and made freely available for the scientific community. The results will have wide applications in statistics and machine learning.Specifically, the use of mini-batches in Markov Chain Monte Carlo (MCMC) will be investigated. MCMC is perhaps the most widely used computational approach for Bayesian statistical inference. Since each step in the simulation of the Markov chain requires the scanning of all the observations, for a large data set this computation is prohibitive. On the other hand, in the area of machine learning researchers have found that stochastic optimization techniques, which examine only a mini-batch of data points at a time, can deliver excellent performance. In this project, a framework for unifying mini-batch based MCMC and global optimization will be developed. It is showed that simulation from of a tempered version of the posterior distribution can be approximated by a MCMC process with Metropolis-Hasting updates that depend only on mini-batches. This approach will be combined with eqi-energy sampling to achieve a unified simulation and global optimization methodology. This framework will allow us to improve the performance of both MCMC methods and non-convex global optimization methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当前统计学和机器学习应用产生的数据集非常大，需要大型模型进行分析。贝叶斯推理和全局优化是从此类数据中学习的两种强大方法，但数据集的大小和由此产生的计算困难极大地限制了这些方法的适用性。 该项目的研究旨在提高这些方法的计算效率，从而大大扩展其在大型数据集分析中的实用性。这项研究的方法和算法将在现代分布式计算平台上实现，并免费提供给科学界。研究结果将在统计学和机器学习中有广泛的应用，特别是在马尔可夫链蒙特卡罗（MCMC）中使用小批量。MCMC可能是贝叶斯统计推断中使用最广泛的计算方法。由于马尔可夫链模拟中的每一步都需要扫描所有的观察结果，因此对于大数据集来说，这种计算是禁止的。另一方面，在机器学习领域，研究人员发现，随机优化技术一次只检查一小批数据点，可以提供出色的性能。在这个项目中，将开发一个统一的小批量MCMC和全局优化的框架。它表明，模拟从一个回火版本的后验分布可以近似MCMC过程与Metropolis-Hasting更新，只依赖于小批量。这种方法将与均衡能量采样相结合，以实现统一的模拟和全局优化方法。该框架将使我们能够提高MCMC方法和非凸全局优化方法的性能。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

An Equation for the Identification of Average Causal Effect in Nonlinear Models

非线性模型中平均因果效应识别方程

• DOI: 10.5705/ss.202021.0191
• 发表时间: 2023
• 期刊: Statistica Sinica
• 影响因子: 1.4
• 作者: Wong, Wing Hung