Developments in Gaussian Processes and Beyond: Applications in Geostatistics and Deep Learning

高斯过程在统计学和机器学习中有不同的应用，是当代的一大兴趣。举几个例子，它们出现在空间数据的建模、计算机实验和研究深度神经网络的极限中。高斯过程吸引人的关键原因包括其简单性和广泛的可追溯性：整个过程的特征仅为均值和协方差函数。然而，尽管高斯过程具有完善的理论和计算特性，但在使用它们时存在一些明显的限制。此外，在一些情况下，高斯过程不适合作为建模选择。将开发新的方法来解决其中的一些限制，从空间统计到深度学习的广泛影响。公开可用的软件开发、学生指导和研究的广泛传播将产生超出手头特定研究问题的影响。技术调查的重点领域如下：第一个问题涉及无处不在的Matern协方差函数的使用。Matern系列的一个关键优点是对高斯过程（GP）实现的平滑度进行精确控制。然而，Matern协方差的尾部呈指数级衰减，这在多项式依赖的情况下是不合适的。多项式协方差（如柯西协方差）弥补了这个问题，但代价是失去对平滑性的控制，因为使用柯西协方差的GP实现要么是无限可微的，要么根本不可微。PI将开发一个新的协方差函数，它结合了Matern和多项式协方差的灵活性。接下来，PI将研究深度神经网络在全局-局部马蹄形正则化先验权下的极限行为。由于缺乏有界矩，需要构建一个新的Levy过程，该过程可用于研究神经网络在这种先验条件下的极限，从而有助于不确定性量化。PI将研究合成过程的理论和计算特性。最后，PI将使用最近开发的全局-局部收缩方法在GP回归中进行贝叶斯正则化，并对现有方法进行明显改进。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Gaussian processes have diverse applications in statistics and machine learning and are of great contemporary interest. To give a few examples, they arise in the modeling of spatial data, computer experiments, and in studying the limits of deep neural networks. Key reasons for the appeal of Gaussian processes include their simplicity and wide tractability: the entire process is characterized by just the mean and the covariance functions. Yet, although Gaussian processes are popular with well-developed theoretical and computational properties, there are some distinct limitations in using them. Moreover, there are several situations where Gaussian processes are inappropriate as a modeling choice. New methodology will be developed to address some of these limitations, with wide-ranging implications from spatial statistics to deep learning. Publicly available software development, student mentoring, and broad dissemination of research will have impacts beyond the particular research problems at hand.Key areas of the technical investigation are as follows. The first issue concerns the use of the ubiquitous Matern covariance function. A key benefit of the Matern family is the precise control over the smoothness of the resultant Gaussian processes (GP) realizations. However, the tails of the Matern covariance decay exponentially fast, which is inappropriate in the presence of polynomial dependence. Polynomial covariances such as Cauchy remedy this issue, but at the expense of a loss of control over smoothness, in that, GP realizations using Cauchy covariances are either infinitely differentiable or not at all. The PI will develop a new covariance function that combines the flexibility of the Matern and polynomial covariances. Next, the PI will study the limiting behavior of deep neural networks under global-local horseshoe regularization priors on the weights. The lack of bounded moments necessitates the construction of a new Levy process that can be used to study the limits of neural networks under such priors, thereby aiding uncertainty quantification. The PI will study the theoretical and computational properties of the resultant process. Finally, the PI will use recently developed global-local shrinkage approaches for Bayesian regularization in GP regression, with distinct improvements upon existing methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.