Collaborative Research: Bayesian Residual Learning and Random Recursive Partitioning Methods for Gaussian Process Modeling

罕见的自然灾害(例如风暴潮和飓风)可造成生命损失和对社会和环境的毁灭性破坏。例如，卡特里娜飓风(2005年)造成新奥尔良地区和密西西比州沿海风暴潮造成1500多人死亡，总损失估计达750亿美元。不确定性量化已被广泛应用于了解、监测和预测这些罕见的自然灾害。高斯过程(GP)建模框架是解决这类UQ应用的最广泛使用的工具之一，已经在几个领域进行了研究，包括空间统计、计算机实验的设计和分析以及机器学习。随着测量技术的进步和计算能力的增强，在现代应用中，大量的测量和不断提高的分辨率的大规模数值模拟被收集起来，这给预测具有相关不确定性的真实世界过程带来了一些关键的挑战。虽然遗传算法为诸如沿海洪水灾害研究等现代新兴应用提供了一条很有前途的执行UQ任务的途径，但现有的遗传算法不能解决一些显著的问题，如由于大数据集的计算瓶颈和由于多维域中复杂结构的空间异质性。该项目将开发新的贝叶斯GP方法，以实现可伸缩计算并捕获空间异质性。新的方法、算法、理论和软件有望改进GP建模，以解决包括物理科学、工程、医学、公共卫生和商业科学在内的广泛领域的数据分析问题。该项目将开发和分发用户友好的开源软件，并为本科生和研究生提供跨学科的研究培训机会。该项目旨在开发一个新的贝叶斯多尺度残差学习框架，该框架具有强大的理论支持，允许可伸缩计算和空间非平稳性用于GP建模。该框架集成和扩展了几种强大的技术，包括预测过程近似、分块收缩和域上的随机递归划分。这一框架将GP分解成一系列残差过程，这些残差过程以不同的分辨率表征基本的协方差结构，并且可以以各种方式在空间上是不同的。新的框架允许采用分块收缩来推断残差过程的协方差，并加入了随机划分先验，以使其能够自适应于多维域中的各种空间结构。在小波收缩和状态空间模型的启发下，将开发新的递归算法，以实现关于观测值数量的线性计算复杂性和线性存储复杂性。所得到的GP方法将保证在串行计算环境中的线性计算复杂性，并且易于并行化。这种贝叶斯多尺度残差学习方法提供了一种新的方法来解决空间统计、计算机实验的设计和分析、机器学习和非参数回归中的GP建模问题。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Rare natural hazards (for example, storm surge and hurricanes) can cause loss of lives and devastating damage to society and the environment. For instance, Hurricane Katrina (2005) caused over 1,500 deaths and total estimated damages of $75 billion in the New Orleans area and along the Mississippi coast as a result of storm surge. Uncertainty quantification (UQ) has been used widely to understand, monitor, and predict these rare natural hazards. The Gaussian process (GP) modeling framework is one of the most widely used tools to address such UQ applications and has been studied across several areas, including spatial statistics, design and analysis of computer experiments, and machine learning. With the advance of measurement technology and increasing computing power, large numbers of measurements and large-scale numerical simulations at increasing resolutions are routinely collected in modern applications and have given rise to several critical challenges in predicting real-world processes with associated uncertainty. While GP presents a promising route to carrying out UQ tasks for modern emerging applications such as coastal flood hazard studies, existing GP methods are inadequate in addressing several notable issues such as computational bottleneck due to big datasets and spatial heterogeneity due to complex structures in multi-dimensional domains. This project will develop new Bayesian GP methods to allow scalable computation and to capture spatial heterogeneity. The new methods, algorithms, theory, and software are expected to improve GP modeling for addressing data analytical issues across a wide range of fields, including physical science, engineering, medical science, public health, and business science. The project will develop and distribute user-friendly open-source software and provide interdisciplinary research training opportunities for undergraduate and graduate students.This project aims to develop a new Bayesian multi-scale residual learning framework with strong theoretical support that allows scalable computation and spatial nonstationarity for GP modeling. This framework integrates and extends several powerful techniques respectively arising in the literature on GP and that on multi-scale modeling, including predictive process approximation, blockwise shrinkage, and random recursive partitioning on the domain. This framework decomposes the GP into a cascade of residual processes that characterize the underlying covariance structures at different resolutions and that can be spatially heterogeneous in a variety of ways. The new framework allows for adoption of blockwise shrinkage to infer the covariance of the residual processes and incorporates random partition priors to enable adaptivity to various spatial structures in multi-dimensional domains. New recursive algorithms inspired by wavelet shrinkage and state-space models will be developed to achieve linear computational complexity and linear storage complexity in terms of the number of observations. The resulting GP method will guarantee linear computational complexity in a serial computing environment and also be easily parallelizable. This Bayesian multi-scale residual learning method provides a new approach to addressing GP modeling issues among spatial statistics, design and analysis of computer experiments, machine learning, and nonparametric regression.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.