Scalable Gaussian-process regression and variable selection using Vecchia approximations
Scalable Gaussian-process regression and variable selection using Vecchia approximations
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
Jian Cao;J. Guinness;M. Genton;M. Katzfuss
中科院分区：
文献类型：
--
作者：
Jian Cao;J. Guinness;M. Genton;M. Katzfuss
高斯过程（GP）回归是一种灵活的非参数回归方法，可以自然量化不确定性。在许多应用中，响应和协变量的数量都很大，目标是选择与响应相关的协变量。对于这种设置，我们提出了一种新的，可扩展的算法，创造VGPR，优化惩罚GP对数似然的基础上的Vecchia GP近似，从空间统计的有序条件近似，这意味着一个稀疏的Cholesky因子的精度矩阵。我们遍历正则化路径从强到弱的惩罚，依次添加候选协变量的对数似然梯度的基础上，并通过一个新的二次约束坐标下降算法取消选择无关的协变量。我们提出了基于Vecchia的小批量子采样，它提供了无偏梯度估计。由此产生的过程是可扩展的数百万个响应和数千个协变量。理论分析和数值研究表明，相对于现有的方法，改进的可扩展性和准确性。
Gaussian process (GP) regression is a flexible, nonparametric approach to regression that naturally quantifies uncertainty. In many applications, the number of responses and covariates are both large, and a goal is to select covariates that are related to the response. For this setting, we propose a novel, scalable algorithm, coined VGPR, which optimizes a penalized GP log-likelihood based on the Vecchia GP approximation, an ordered conditional approximation from spatial statistics that implies a sparse Cholesky factor of the precision matrix. We traverse the regularization path from strong to weak penalization, sequentially adding candidate covariates based on the gradient of the log-likelihood and deselecting irrelevant covariates via a new quadratic constrained coordinate descent algorithm. We propose Vecchia-based mini-batch subsampling, which provides unbiased gradient estimators. The resulting procedure is scalable to millions of responses and thousands of covariates. Theoretical analysis and numerical studies demonstrate the improved scalability and accuracy relative to existing methods.