Developments in Gaussian Processes and Beyond: Applications in Geostatistics and Deep Learning

高斯过程在统计学和机器学习中有着广泛的应用，并在当代引起了极大的兴趣。举几个例子，它们出现在空间数据的建模、计算机实验和研究深度神经网络的局限性中。高斯过程吸引人的主要原因包括它们的简单性和广泛的可操作性：整个过程的特征只有均值和协方差函数。然而，尽管高斯过程具有良好的理论和计算特性而广受欢迎，但在使用它们时存在一些明显的限制。此外，在某些情况下，高斯过程不适合作为建模选择。将开发新的方法来解决其中一些限制，涉及从空间统计到深度学习的广泛影响。公开可用的软件开发、学生指导和研究成果的广泛传播将产生超出手头特定研究问题的影响。技术调查的关键领域如下。第一个问题涉及无处不在的马特恩协方差函数的使用。Matn系列的一个主要优点是可以精确控制所得到的高斯过程(GP)实现的光滑度。然而，Matn协方差的尾部以指数速度衰减，这在存在多项式依赖的情况下是不合适的。多项式协方差(如柯西协方差)解决了这个问题，但代价是失去了对光滑性的控制，因为使用柯西协方差的GP实现要么是无限可微的，要么根本不是。PI将开发一种新的协方差函数，它结合了MATRUN和多项式协方差的灵活性。下一步，PI将研究全局-局部马蹄形正则化先验关于权值的深度神经网络的极限行为。由于缺乏有界矩，需要构造一个新的Levy过程，该过程可用于研究这种先验条件下神经网络的极限，从而有助于不确定性的量化。PI将研究合成过程的理论和计算特性。最后，PI将使用最近开发的全局-局部收缩方法在GP回归中进行贝叶斯正则化，并对现有方法进行了显著改进。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Gaussian processes have diverse applications in statistics and machine learning and are of great contemporary interest. To give a few examples, they arise in the modeling of spatial data, computer experiments, and in studying the limits of deep neural networks. Key reasons for the appeal of Gaussian processes include their simplicity and wide tractability: the entire process is characterized by just the mean and the covariance functions. Yet, although Gaussian processes are popular with well-developed theoretical and computational properties, there are some distinct limitations in using them. Moreover, there are several situations where Gaussian processes are inappropriate as a modeling choice. New methodology will be developed to address some of these limitations, with wide-ranging implications from spatial statistics to deep learning. Publicly available software development, student mentoring, and broad dissemination of research will have impacts beyond the particular research problems at hand.Key areas of the technical investigation are as follows. The first issue concerns the use of the ubiquitous Matern covariance function. A key benefit of the Matern family is the precise control over the smoothness of the resultant Gaussian processes (GP) realizations. However, the tails of the Matern covariance decay exponentially fast, which is inappropriate in the presence of polynomial dependence. Polynomial covariances such as Cauchy remedy this issue, but at the expense of a loss of control over smoothness, in that, GP realizations using Cauchy covariances are either infinitely differentiable or not at all. The PI will develop a new covariance function that combines the flexibility of the Matern and polynomial covariances. Next, the PI will study the limiting behavior of deep neural networks under global-local horseshoe regularization priors on the weights. The lack of bounded moments necessitates the construction of a new Levy process that can be used to study the limits of neural networks under such priors, thereby aiding uncertainty quantification. The PI will study the theoretical and computational properties of the resultant process. Finally, the PI will use recently developed global-local shrinkage approaches for Bayesian regularization in GP regression, with distinct improvements upon existing methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.