Sampling from Distributions with Intractable Integrals

在过去的五十年中，马尔可夫链蒙特卡罗（MCMC）方法已经发展成为一个通用的和强大的工具，科学计算。 然而，正如许多研究人员所知，传统的MCMC方法无法从具有棘手积分的分布中采样。这个项目的目标是开发一些创新的蒙特卡罗算法，这些算法能够从具有难处理积分的分布中进行采样。为了实现这一目标，PI提出了一种新的群体蒙特卡罗算法--蒙特卡罗动态加权重要性抽样（MCDWIS）。在模拟中，MCDWIS通过其蒙特卡罗估计来替换难处理积分的比率，并且通过对产生的新样本赋予不同的权重来抵消由此引入的偏差。MCDWIS允许在MCMC模拟中使用Monte Carlo估计，同时使目标分布相对于重要权重保持不变。与辅助变量MCMC方法不同，MCDWIS避免了对完美样本的要求，因此可以应用于许多统计模型，完美的采样是不可用的或非常昂贵的。如提案中所讨论的，MCDWIS还可以用于从缺失数据和随机效应相关模型的不完全后验分布中进行采样（例如，广义线性混合模型），传统上用期望最大化（EM）或Monte Carlo EM算法处理。除了为这些模型提供完全的贝叶斯分析外，MCDWIS还可以克服EM和Monte Carlo EM算法所遇到的局部陷阱问题，这是由于其自调整机制。在该提案中，PI还提出了一种重要性抽样目标随机近似蒙特卡罗算法，即所谓的重要性随机近似蒙特卡罗算法，该算法可用于对具有难以处理的归一化常数的模型进行贝叶斯推断。该项目的智力价值在于提供了一些创新的计算方法，这些模型被期望在一类重要的科学模型的统计推断中发挥主要作用，包括用于社会网络分析的随机图模型，用于空间数据分析的自规范模型，用于疾病绘图的自逻辑模型，以及用于生物医学数据分析的广义线性混合模型等。模型的成功推理将增强人们对潜在的自然、社会或生物系统的理解。该项目将在统计方法和科学计算领域产生更广泛的影响。研究成果将通过与其他学科研究人员的直接合作、会议演示、书籍和在学术期刊上发表的论文传播给这些社区。 该项目还将通过研究生直接参与项目并将成果纳入本科生和研究生课程，对教育产生重大影响。

During the past five decades, Markov chain Monte Carlo (MCMC) methods have been developed as a versatile and powerful tool for scientific computing. However, as known by many researchers, conventional MCMC methods suffer from the inability to sample from distributions with intractable integrals. The goal of this project is to develop some innovative Monte Carlo algorithms which are capable of sampling from distributions with intractable integrals. To achieve this goal, the PI proposes a new population Monte Carlo algorithm---Monte Carlo dynamically weighted importance sampling (MCDWIS). In simulations, MCDWIS replaces the ratio of intractable integrals by its Monte Carlo estimate, and the bias introduced thereby is counterbalanced by giving different weights to new samples produced. MCDWIS allows for the use of Monte Carlo estimates in MCMC simulations, while leaving the target distribution invariant with respect to important weights. Unlike auxiliary variable MCMC methods, MCDWIS avoids the requirement for perfect samples, and thus can be applied to many statistical models for which perfect sampling is unavailable or very expensive. As discussed in the proposal, MCDWIS can also be used to sample from incomplete posterior distributions for missing data and random effects-related models (e.g., generalized linear mixed models), which are traditionally treated with the expectation-maximization (EM) or Monte Carlo EM algorithms. In addition to providing a fully Bayesian analysis for these models, the MCDWIS can potentially overcome, due to its self-adjusting mechanism, the local-trap problem suffered by the EM and Monte Carlo EM algorithms. In this proposal, the PI also proposes an importance sampling-targeted stochastic approximation Monte Carlo algorithm, the so-called importance stochastic approximation Monte Carlo algorithm, which can be used for Bayesian inference for the models with intractable normalizing constants.The intellectual merit of this project is to provide some innovative computational methods, which are expected to play a major role in statistical inference for an important class of scientific models, including random graph models used in social network analysis, autonormal models used in spatial data analysis, autologistic models used in disease mapping, and generalized linear mixed models used in biomedical data analysis, among others. Successful inferences of the models will enhance people's underderstanding to the underlying natural, social, or biological systems. This project will have broader impacts in both communities of statistical methodology and scientific computing. The research results will be disseminated to these communities via direct collaboration with researchers in other disciplines, conference presentations, books, and papers to be published in academic journals. The project will have also significant impacts on education through direct involvement of graduate students in the project and incorporation of results into undergraduate and graduate courses.