CAREER: Unbiased Estimation with Faithful Markov Chains for Scalable Statistical Inference

数值积分是所有需要模拟和校准复杂概率模型的科学领域的共同目标。在统计学中，从参数推断到模型平均和假设检验，几乎所有的任务都使用数值积分。在最先进的数值积分技术中，大多数方法是迭代操作的随机算法，一个接一个地生成随机状态序列。不幸的是，它们的迭代性质与当前计算硬件的方向不一致：日益并行的体系结构和停滞不前的时钟频率。这项研究开发了新的算法，可以提供积分的准确估计，因为可以独立和并行生成的随机量的数量达到无穷大。建议的技术被用来解决大型模型和复杂数据的统计推断中的长期挑战。这些创新结合了应用概率、计算机科学和统计计算，适用于机器学习、统计力学、计算神经科学和流行病学等高维积分丰富的领域。该项目涉及软件的开发，并以为学生提供课程和研究机会的教育计划为特色，以及更广泛的传播计划。为了数值逼近高维积分，马尔可夫链蒙特卡罗方法迭代地生成探索被积函数所描述的景观的序列。这些方法产生的估计器在迭代次数的限制下收敛于感兴趣的积分。然而，在并行计算硬件时代，依赖于迭代渐近机制的算法可能会过时。这项拟议的研究开发了新的蒙特卡罗估计器，这种估计器对兴趣的预期是无偏的，同时具有有限的计算成本和有限的方差。因此，它们可以独立地并行生成并求平均，为大规模并行计算机上的可扩展数值积分铺平了道路。所提出的估计器依赖于马尔可夫链的忠实耦合，即在随机次数的迭代之后，链对合并。该项目包括对所提出的估计器的效率的理论研究，以及针对各种应用的实用耦合策略的设计。这项研究与数值方法、随机过程和最优运输等主题有关。除了并行计算，建议的估计器还用于解决统计挑战，如对由多个组件组成的大型模型的常量估计和模块化推理进行标准化。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Numerical integration is a common goal in all scientific fields where complex probabilistic models need to be simulated and calibrated. In statistics, numerical integration is used in virtually all tasks, from parameter inference to model averaging and hypothesis testing. Among state-of-the-art numerical integration techniques, most methods are randomized algorithms that operate iteratively, generating a sequence of random states, one after the other. Unfortunately, their iterative nature stands at odds with current directions in computing hardware: increasingly parallel architectures and stagnating clock rates. This research develops new algorithms that provide accurate estimates of integrals as a number of random quantities, that can be generated independently and in parallel, goes to infinity. The proposed techniques are employed to address long-standing challenges in statistical inference for large models and complex data. The proposed innovations combine applied probability, computer science and statistical computing, and apply to many fields including machine learning, statistical mechanics, computational neuroscience and epidemiology, where high-dimensional integrals abound. The project involves the development of software and features an educational program with courses and research opportunities for students, and a broader dissemination program. To numerically approximate high-dimensional integrals, Markov Chain Monte Carlo methods iteratively generate sequences that explore the landscape described by the integrand. These methods yield estimators that converge to the integrals of interest in the limit of the number of iterations. However, algorithms that rely on iterative asymptotic regimes risk becoming obsolete in the era of parallel computing hardware. The proposed research develops new Monte Carlo estimators that are unbiased for the expectations of interest, while having a finite computing cost and a finite variance. They can thus be generated independently in parallel and averaged over, paving the way for scalable numerical integration on large-scale parallel computers. The proposed estimators rely on faithful couplings of Markov chains, whereby pairs of chains coalesce after a random number of iterations. This project includes theoretical investigations on the efficiency of the proposed estimators, and the design of practical coupling strategies for various applications. The research connects with topics in numerical methods, stochastic processes and optimal transport. Beyond parallel computing, the proposed estimators are used to tackle statistical challenges such as normalizing constant estimation and modular inference for large models made of multiple components.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.