CAREER: Towards Tight Guarantees of Markov Chain Sampling Algorithms in High Dimensional Statistical Inference

从分布中提取样本是贝叶斯统计、机器学习、统计物理和许多其他涉及随机模型的领域中的核心计算挑战。在所有的方法中，马尔可夫链蒙特卡罗(MCMC)算法是应用最广泛的一类抽样算法，尤其是在高维贝叶斯推理中。而自Metropolis等人的基础性工作以来，已经提出、研究和实现了MCMC算法。1953年，人们对实际使用的算法的许多收敛性质还没有很好的了解。贝叶斯统计的实践者经常面临一系列需要严格解决的关键挑战：算法超参数的选择、估计计算成本和最佳算法的选择等。本项目专注于开发大规模贝叶斯统计推理问题中出现的MCMC抽样算法的理论保证。该项目还将为从本科生到博士生的下一代统计学家和数据科学家提供大量跨学科的研究培训、拓展和指导机会。本项目将解决高维推理中围绕MCMC算法的三个具体研究问题。首先，该项目打算严格地对采样对数凹分布的MCMC算法的效率进行排名，并提供关于混合时间的简洁的非渐近极大极小分析。对数凹分布在抽样中与凸函数在最优化中同样重要，如果不确定对数凹分布抽样算法的基本极限，就不能期望建立一个基本的理论基础。哈密尔顿蒙特卡罗、吉布斯抽样和肇事逃逸等广泛使用的算法将被严格研究。其次，由于浓度不等是理解MCMC抽样算法效率的重要组成部分，该项目将通过随机局部化等新技术工具发展对高维对数凹分布的浓度的细粒度理解。最后，该项目将通过本地化方案统一研究离散状态和连续状态采样算法的现有理论工具。这项拟议的研究旨在通过全面了解MCMC采样算法及其在离散和连续情况下的最佳设置来推动该领域的发展。该项目将提供广泛的跨学科倡议，以促进统计科学本科生和研究生的专业发展。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Drawing samples from a distribution is a core computational challenge in fields such as Bayesian statistics, machine learning, statistical physics, and many other areas involving stochastic models. Among all methods, Markov Chain Monte Carlo (MCMC) algorithms stand out as the most widely used class of sampling algorithms with a broad range of applications, notably in high dimensional Bayesian inference. While MCMC algorithms have been proposed, studied, and implemented since the foundational work of Metropolis et al. in 1953, many convergence properties of algorithms used in practice are not well understood. Practitioners in Bayesian statistics are often faced with a series of key challenges to be addressed rigorously: the choice of algorithm hyper-parameters, the estimated computational cost and the choice of the best algorithm, etc. This project focuses on developing theoretical guarantees of MCMC sampling algorithms that arise in large-scale Bayesian statistical inference problems.The project will also offer numerous interdisciplinary research training, outreach and mentoring opportunities for the next generation of statisticians and data scientists at all levels, from undergraduate to doctoral students.This project will address three specific research problems centered around MCMC algorithms in high dimensional inference. First, the project intends to rigorously rank the efficiency of MCMC algorithms for sampling log-concave distributions and to provide succinct non-asymptotic mini-max analysis of mixing time. Log-concave distributions in sampling are as important as convex functions in optimization, and one cannot expect to build a foundational theory basis without determining the fundamental limits of sampling algorithms on log-concave distributions. Widely-used algorithms such as Hamiltonian Monte Carlo, Gibbs sampling and hit-and-run will be studied rigorously. Second, as concentration inequalities constitute an essential component in understanding the efficiency of MCMC sampling algorithms, the project will develop a fine-grained understanding of concentration of high dimensional log-concave distributions via new technical tools such as stochastic localization. Finally, the project will unify the existing theoretical tools for studying discrete-state and continuous-state sampling algorithms through localization schemes. The proposed research aims to advance the field with a comprehensive understanding of MCMC sampling algorithms and their optimal settings in both discrete and continuous cases. The project will provide a wide range of interdisciplinary initiatives to enhance professional development of undergraduate and graduate students in statistical sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.