Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation
Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation
复制标题
分层参数估计的经验贝叶斯和核流的一致性
DOI：
10.1090/mcom/3649
复制
发表时间：
2021
影响因子：
2
通讯作者：
Stuart, Andrew M.
中科院分区：
文献类型：
--
作者：
Chen, Yifan;Owhadi, Houman;Stuart, Andrew M.
高斯过程回归在统计学、机器学习和反问题中已被证明是非常强大的。在对复杂和现实世界问题的广泛应用中，这种方法成功的一个关键方面是超参数的分层建模和学习。本文的目的是研究两种学习分层参数的范例：一种是从概率贝叶斯的角度，特别是贝叶斯统计中广泛使用的经验贝叶斯方法；另一种是从确定性和逼近理论的角度，特别是最近在机器学习文献中提出的核流算法。本文针对环面上的类Matérn模型，建立了它们在大数据极限下的一致性分析，以及它们在参数学习中的隐含偏差的显式辨识。我们克服的一个特别的技术挑战是学习类Matérn场中的正则性参数，在空间统计文献中，关于这一点的一致性结果一直非常少。此外，我们还在类Matérn模型之外进行了大量的数值实验，进一步比较了两种算法的优劣。这些实验演示了其他层次参数的学习，如幅度和长度尺度；它们也说明了模型误指定的设置，在这种情况下，核流方法可以表现出比更传统的经验贝叶斯方法更好的性能。参考文献
Gaussian process regression has proven very powerful in statistics, machine learning and inverse problems. A crucial aspect of the success of this methodology, in a wide range of applications to complex and real-world problems, is hierarchical modeling and learning of hyperparameters. The purpose of this paper is to study two paradigms of learning hierarchical parameters: one is from the probabilistic Bayesian perspective, in particular, the empirical Bayes approach that has been largely used in Bayesian statistics; the other is from the deterministic and approximation theoretic view, and in particular the kernel flow algorithm that was proposed recently in the machine learning literature. Analysis of their consistency in the large data limit, as well as explicit identification of their implicit bias in parameter learning, are established in this paper for a Matérn-like model on the torus. A particular technical challenge we overcome is the learning of the regularity parameter in the Matérn-like field, for which consistency results have been very scarce in the spatial statistics literature. Moreover, we conduct extensive numerical experiments beyond the Matérn-like model, comparing the two algorithms further. These experiments demonstrate learning of other hierarchical parameters, such as amplitude and lengthscale; they also illustrate the setting of model misspecification in which the kernel flow approach could show superior performance to the more traditional empirical Bayes approach. References