CAREER: Towards Tight Guarantees of Markov Chain Sampling Algorithms in High Dimensional Statistical Inference

从分布中提取样本是贝叶斯统计、机器学习、统计物理和许多其他涉及随机模型的领域的核心计算挑战。在所有方法中，马尔可夫链蒙特卡罗（MCMC）算法是使用最广泛的一类采样算法，具有广泛的应用范围，特别是在高维贝叶斯推理中。虽然自Metropolis等人1953年的基础工作以来，MCMC算法已经被提出、研究和实现，但在实践中使用的算法的许多收敛特性还没有得到很好的理解。贝叶斯统计的实践者经常面临一系列需要严格解决的关键挑战：算法超参数的选择、估计的计算成本和最佳算法的选择等。该项目侧重于开发大规模贝叶斯统计推断问题中出现的MCMC采样算法的理论保证。该项目还将为从本科生到博士生的各级下一代统计学家和数据科学家提供众多跨学科研究培训、外展和指导机会。本项目将围绕高维推理中的MCMC算法解决三个具体的研究问题。首先，该项目打算对采样对数凹分布的MCMC算法的效率进行严格排序，并提供简洁的混合时间的非渐近最小最大值分析。采样中的对数凹分布与优化中的凸函数一样重要，如果不确定对数凹分布上采样算法的基本限制，就不能指望建立基本的理论基础。广泛使用的算法，如哈密顿蒙特卡罗，吉布斯采样和肇事逃逸将被严格研究。其次，由于浓度不等式是理解MCMC采样算法效率的重要组成部分，该项目将通过随机定位等新技术工具，对高维对数凹分布的浓度进行细粒度理解。最后，该项目将通过定位方案统一现有的研究离散状态和连续状态采样算法的理论工具。提出的研究旨在通过全面理解MCMC采样算法及其在离散和连续情况下的最佳设置来推进该领域。该计划将提供广泛的跨学科活动，以加强统计科学本科生和研究生的专业发展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Drawing samples from a distribution is a core computational challenge in fields such as Bayesian statistics, machine learning, statistical physics, and many other areas involving stochastic models. Among all methods, Markov Chain Monte Carlo (MCMC) algorithms stand out as the most widely used class of sampling algorithms with a broad range of applications, notably in high dimensional Bayesian inference. While MCMC algorithms have been proposed, studied, and implemented since the foundational work of Metropolis et al. in 1953, many convergence properties of algorithms used in practice are not well understood. Practitioners in Bayesian statistics are often faced with a series of key challenges to be addressed rigorously: the choice of algorithm hyper-parameters, the estimated computational cost and the choice of the best algorithm, etc. This project focuses on developing theoretical guarantees of MCMC sampling algorithms that arise in large-scale Bayesian statistical inference problems.The project will also offer numerous interdisciplinary research training, outreach and mentoring opportunities for the next generation of statisticians and data scientists at all levels, from undergraduate to doctoral students.This project will address three specific research problems centered around MCMC algorithms in high dimensional inference. First, the project intends to rigorously rank the efficiency of MCMC algorithms for sampling log-concave distributions and to provide succinct non-asymptotic mini-max analysis of mixing time. Log-concave distributions in sampling are as important as convex functions in optimization, and one cannot expect to build a foundational theory basis without determining the fundamental limits of sampling algorithms on log-concave distributions. Widely-used algorithms such as Hamiltonian Monte Carlo, Gibbs sampling and hit-and-run will be studied rigorously. Second, as concentration inequalities constitute an essential component in understanding the efficiency of MCMC sampling algorithms, the project will develop a fine-grained understanding of concentration of high dimensional log-concave distributions via new technical tools such as stochastic localization. Finally, the project will unify the existing theoretical tools for studying discrete-state and continuous-state sampling algorithms through localization schemes. The proposed research aims to advance the field with a comprehensive understanding of MCMC sampling algorithms and their optimal settings in both discrete and continuous cases. The project will provide a wide range of interdisciplinary initiatives to enhance professional development of undergraduate and graduate students in statistical sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.