Next-generation Tempering Methods for Multimodal Sampling: Theory and Applications

我们今天收集的数据比以往任何时候都更加复杂，需要复杂的统计模型来发现复杂的模式并提取有用的信息。这些模型有不同的形式，例如用于图像和语音识别的神经网络模型，用于文本分类的混合模型，以及用于研究基因之间相互作用的图形模型。但随着数据量和复杂性的不断增加，用这样的模型进行精确计算是不可行的。数据科学中普遍使用的一种可行的替代方法是抽样，它可以生成近似的、随机的和计算高效的解。例如，给定一个包含数千个基因的数据集，通过根据某种抽样方案重复生成随机基因网络，可以确定哪个网络最好地解释了观察到的数据，并量化了基因相互作用的概率。然而，抽样方法的性能在不同的问题上可能会有很大的差异，而且通常很难从理论上进行分析。在这个项目中，将开发一种具有理论保障的范式转换抽样方法。所开发的方法具有广泛的适用性，当存在大量的好的解决方案时尤其有用，但传统的抽样算法只能识别其中的几个。研究小组将实施解决基因组学中计算密集型问题的方法，如复杂性状的遗传力估计，以及研究不同棉花品种的水分利用效率。后者将帮助农学家开发和选择抗旱性更好的高产棉花品种。这项研究包括适合培养研究生和开源软件开发的项目。该项目旨在开发从多峰目标分布中抽样的新算法，这是统计学、机器学习和应用科学中普遍存在的计算挑战。提出了一种设计马尔可夫链蒙特卡罗方法的新方法，该方法将现有的抽样技术联系起来，包括随机游走的Metropolis-Hastings算法、重要性抽样和模拟回火。这一新的范式使研究人员能够方便地结合不同的技术，导致理论和方法上的进步，这在经典框架中是不可能的。特别是，研究团队将开发涉及多个链的新算法，根据链是用于勘探还是开采来优化每个链的行为和计算成本。收敛分析将在一般的多模式设置下进行，以提供理论保障。将深入研究的拟议抽样方法的一个应用是偏微分方程模型的学习。提出了一种新的基于高斯过程的贝叶斯方法，用于推断高度非线性偏微分方程模型中的未知参数，该方法可以与所提出的抽样方法无缝集成。还将研究其他应用，包括高维模型选择问题，并开发针对每个问题的算法。所有由此产生的软件包都将公开提供。最终，在这个项目中开发的统计方法和算法将被用来解决基因组学和作物科学中的紧迫问题。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The data we collect today is more complex than ever, requiring sophisticated statistical models to uncover intricate patterns and extract useful information. These models come in various forms, such as neural network models for image and speech recognition, mixture models for text classification, and graphical models for studying the interaction between genes. But the increasing volume and complexity of the data make it infeasible to perform exact calculations with such models. A viable alternative used ubiquitously in data science is sampling, which can generate approximate, randomized, and computationally efficient solutions. For example, given a data set of thousands of genes, by generating random gene networks repeatedly according to some sampling scheme, one can determine which network best explains the observed data and quantify the probabilities of gene interactions. However, the performance of sampling methods can vary significantly across different problems and is often hard to analyze theoretically. In this project, a paradigm-shifting sampling methodology with theoretical guarantees will be developed. The methods developed are widely applicable and are particularly useful when a large number of good solutions to the problem exist, but only a few can be identified by traditional sampling algorithms. The research team will implement the methods for solving computationally intensive problems in genomics, such as heritability estimation of complex traits, and for studying the water use efficiency of different cotton varieties. The latter will help agronomists develop and select high-yielding cotton varieties with better drought tolerance. The research includes projects suitable for training graduate students and open-source software development.This project aims to develop new algorithms for sampling from multimodal target distributions, a universal computational challenge in statistics, machine learning, and applied sciences. A novel approach to devising Markov chain Monte Carlo methods will be developed, which bridges existing sampling techniques, including random walk Metropolis-Hastings algorithms, importance sampling, and simulated tempering. This new paradigm enables researchers to combine different techniques conveniently, leading to both theoretical and methodological advancements that are impossible in the classical framework. Particularly, the research team will develop new algorithms that involve multiple chains, with each chain’s behavior and computational cost optimized according to whether it is used for exploration or exploitation. Convergence analysis will be conducted under general multimodal settings to provide theoretical guarantees. One application of the proposed sampling methodology that will be researched in depth is the learning of partial differential equation models. A novel Bayesian methodology based on Gaussian processes will be developed for inferring unknown parameters in highly non-linear partial differential equation models, which can be seamlessly integrated with the proposed sampling methods. Other applications, including high-dimensional model selection problems, will also be studied, and algorithms tailored to each problem will be developed. All resulting software packages will be made publicly available. Ultimately, the statistical methodology and algorithms developed in this project will be utilized to tackle pressing problems in genomics and crop sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.