CAREER: Unbiased Estimation with Faithful Markov Chains for Scalable Statistical Inference

职业：使用忠实马尔可夫链进行无偏估计，以实现可扩展的统计推断

• 批准号: 1844695
• 负责人: Pierre Jacob
• 金额: $ 40万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1844695&HistoricalAwards=false
• 关键词: CAREER; Unbiased; Estimation; Faithful; Markov

Numerical integration is a common goal in all scientific fields where complex probabilistic models need to be simulated and calibrated. In statistics, numerical integration is used in virtually all tasks, from parameter inference to model averaging and hypothesis testing. Among state-of-the-art numerical integration techniques, most methods are randomized algorithms that operate iteratively, generating a sequence of random states, one after the other. Unfortunately, their iterative nature stands at odds with current directions in computing hardware: increasingly parallel architectures and stagnating clock rates. This research develops new algorithms that provide accurate estimates of integrals as a number of random quantities, that can be generated independently and in parallel, goes to infinity. The proposed techniques are employed to address long-standing challenges in statistical inference for large models and complex data. The proposed innovations combine applied probability, computer science and statistical computing, and apply to many fields including machine learning, statistical mechanics, computational neuroscience and epidemiology, where high-dimensional integrals abound. The project involves the development of software and features an educational program with courses and research opportunities for students, and a broader dissemination program. To numerically approximate high-dimensional integrals, Markov Chain Monte Carlo methods iteratively generate sequences that explore the landscape described by the integrand. These methods yield estimators that converge to the integrals of interest in the limit of the number of iterations. However, algorithms that rely on iterative asymptotic regimes risk becoming obsolete in the era of parallel computing hardware. The proposed research develops new Monte Carlo estimators that are unbiased for the expectations of interest, while having a finite computing cost and a finite variance. They can thus be generated independently in parallel and averaged over, paving the way for scalable numerical integration on large-scale parallel computers. The proposed estimators rely on faithful couplings of Markov chains, whereby pairs of chains coalesce after a random number of iterations. This project includes theoretical investigations on the efficiency of the proposed estimators, and the design of practical coupling strategies for various applications. The research connects with topics in numerical methods, stochastic processes and optimal transport. Beyond parallel computing, the proposed estimators are used to tackle statistical challenges such as normalizing constant estimation and modular inference for large models made of multiple components.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数值积分是所有需要模拟和校准复杂概率模型的科学领域的共同目标。 在统计学中，数值积分几乎用于所有任务，从参数推断到模型平均和假设检验。 在最先进的数值积分技术中，大多数方法都是迭代操作的随机算法，一个接一个地生成随机状态序列。不幸的是，它们的迭代性质与计算硬件的当前方向不一致：越来越多的并行架构和停滞的时钟速率。 这项研究开发了新的算法，提供了精确的估计积分作为一个随机量，可以独立和并行生成，走向无穷大。 所提出的技术用于解决大型模型和复杂数据的统计推断中的长期挑战。 所提出的创新联合收割机结合了应用概率、计算机科学和统计计算，并适用于许多领域，包括机器学习、统计力学、计算神经科学和流行病学，其中高维积分比比皆是。 该项目涉及软件的开发，并具有为学生提供课程和研究机会的教育计划，以及更广泛的传播计划。为了在数值上近似高维积分，马尔可夫链蒙特卡罗方法迭代地生成探索被积函数所描述的景观的序列。这些方法产生的估计收敛到积分的迭代次数的限制。然而，依赖于迭代渐近机制的算法在并行计算硬件时代有过时的风险。拟议的研究开发新的蒙特卡罗估计是无偏的期望的利益，同时具有有限的计算成本和有限的方差。因此，它们可以独立地并行生成并平均，为大规模并行计算机上的可扩展数值积分铺平了道路。建议的估计依赖于忠实的耦合马尔可夫链，从而对链合并后，随机次数的迭代。该项目包括对所提出的估计器的效率进行理论研究，以及为各种应用设计实际的耦合策略。 该研究与数值方法，随机过程和最佳运输的主题有关。除了并行计算之外，拟议的估计器还用于应对统计挑战，例如标准化常数估计和由多个组件组成的大型模型的模块化推断。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

项目成果

Coupling‐based convergence assessment of some Gibbs samplers for high‐dimensional Bayesian regression with shrinkage priors

• DOI: 10.1111/rssb.12495
• 发表时间: 2020-12
• 期刊: Journal of the Royal Statistical Society: Series B (Statistical Methodology)
• 作者: N. Biswas;A. Bhattacharya;P. Jacob;J. Johndrow

Estimating Convergence of Markov chains with L-Lag Couplings

• 发表时间: 2019-05
• 作者: N. Biswas;P. Jacob;Paul Vanetti

Unbiased Markov chain Monte Carlo methods with couplings

• DOI: 10.1111/rssb.12336
• 发表时间: 2020-05-06
• 期刊: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• 影响因子: 5.8
• 作者: Jacob, Pierre E.;O'Leary, John;Atchade, Yves F.
• 通讯作者: Atchade, Yves F.

Maximal Couplings of the Metropolis-Hastings Algorithm

Metropolis-Hastings 算法的最大耦合

• 发表时间: 2021
• 期刊: The 24th International Conference on Artificial Intelligence and Statistics
• 作者: Wang, Guanyang;O’Leary, John;Jacob, Pierre E
• 通讯作者: Jacob, Pierre E

Unbiased Hamiltonian Monte Carlo with couplings

• DOI: 10.1093/biomet/asy074
• 发表时间: 2017-09
• 期刊: Biometrika
• 影响因子: 2.7
• 作者: J. Heng;P. Jacob