Advances in Scalable Monte Carlo Algorithms for Bayesian Statistics

大型数据集和复杂模型的统计分析需求巨大，其应用遍及自然科学和社会科学、工程、商业和生物医学。 在满足这一需求的主要限制在于当前的计算算法，它不能充分扩展到大的问题和数据集。 与此同时，计算机处理器速度的进步已经大幅放缓，促使计算机硬件行业转向并行化。 因此，对贝叶斯计算算法的需求正在迅速增加，因为它们背后的平台正在发生变化。 本文探讨了几个有前途的新方向，以产生高效和可扩展的算法，以及相应的软件工具，适用于通用贝叶斯计算。这项工作涉及贝叶斯计算的新方向，包括（1）通用马尔可夫链蒙特卡罗采样器的真正并行化，一类传统上被视为“固有串行”的算法，（2）“静态数据”基于粒子的顺序蒙特卡罗采样器的算法和理论进展，这些采样器是高度可并行的，但目前在许多复杂的高维后验分布上失败，和（3）新的工具，经验监测的收敛性的Monte Carlo采样器专门设计的复杂分布和高维问题域。 工作的所有方面都直接受到贝叶斯统计在化学动力学，结构生物信息学和系统生物学中的应用的启发。 这项工作也有直接适用于统计物理，计算机科学和分子模拟等更广泛领域的问题。

Statistical analysis of large datasets and complex models is in tremendous demand, with applications throughout the natural and social sciences, engineering, business, and biomedicine. A major limitation in meeting this demand lies in current computational algorithms, which fail to scale adequately to large problems and data sets. At the same time, advances in computer processor speeds have slowed dramatically, prompting a shift in the computer hardware industry towards parallelization. As a result, the demands on Bayesian computational algorithms are increasing rapidly as the platforms underlying them are changing. This work explores several promising new directions for producing highly efficient and scalable algorithms, and corresponding software tools, suitable for general-purpose Bayesian calculations.The work involves new directions in Bayesian computation, including (1) true parallelization of general-purpose Markov chain Monte Carlo samplers, a class of algorithms traditionally viewed as "inherently serial," (2) algorithmic and theoretical advances in "static data" particle-based sequential Monte Carlo samplers which are highly parallelizable but currently fail on many complex high-dimensional posterior distributions, and (3) new tools for empirical monitoring of convergence of Monte Carlo samplers specifically designed for complex distributions and high-dimensional problem domains. All facets of the work are directly motivated by applications of Bayesian statistics in chemical kinetics, structural bioinformatics, and systems biology. This work also has immediate applicability to problems in broader areas of statistical physics, computer science, and molecular simulation.