Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach
Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach
复制标题
DOI：
10.1109/tpami.2020.2987482
复制
发表时间：
2019-01
影响因子：
23.6
通讯作者：
R. Kontar;Garvesh Raskutti;Shiyu Zhou
中科院分区：
文献类型：
--
作者：
R. Kontar;Garvesh Raskutti;Shiyu Zhou
近年来，多元高斯过程（MGP）得到了越来越多的关注，它扩展了高斯过程（GP）来处理多个输出。一种构造MGP并考虑输出之间的非平凡共性的方法采用卷积过程（CP）。CP基于在多个卷积中共享潜在函数的思想。尽管CP结构优雅，但它提供了需要解决的新挑战。首先，即使有中等数量的输出，由于计算需求和要估计的参数数量的巨大增加，模型构建也是极其禁止的。第二，当某些产出没有共同点时，可能会发生知识的负转移。在本文中，我们解决这些问题。我们提出了一个正则化的成对建模方法建立使用CP的MGP。我们的方法的关键特征是将完整的多变量模型的估计分布到一组单独构建的双变量GP中。有趣的是，成对建模具有独特的特征，这使我们能够通过惩罚促进每个双变量模型中信息共享的潜在函数来解决负迁移的挑战。然后通过在贝叶斯框架内组合来自双变量模型的预测来进行预测。该方法具有良好的可扩展性，当输出的数量是大的，并最大限度地减少不相关的输出之间的知识的负转移。所提出的方法的统计保证进行了研究，并通过数值研究证明了其有利的功能。
Recently there has been an increasing interest in the multivariate Gaussian process (MGP) which extends the Gaussian process (GP) to deal with multiple outputs. One approach to construct the MGP and account for non-trivial commonalities amongst outputs employs a convolution process (CP). The CP is based on the idea of sharing latent functions across several convolutions. Despite the elegance of the CP construction, it provides new challenges that need yet to be tackled. First, even with a moderate number of outputs, model building is extremely prohibitive due to the huge increase in computational demands and number of parameters to be estimated. Second, the negative transfer of knowledge may occur when some outputs do not share commonalities. In this paper we address these issues. We propose a regularized pairwise modeling approach for the MGP established using CP. The key feature of our approach is to distribute the estimation of the full multivariate model into a group of bivariate GPs which are individually built. Interestingly pairwise modeling turns out to possess unique characteristics, which allows us to tackle the challenge of negative transfer through penalizing the latent function that facilitates information sharing in each bivariate model. Predictions are then made through combining predictions from the bivariate models within a Bayesian framework. The proposed method has excellent scalability when the number of outputs is large and minimizes the negative transfer of knowledge between uncorrelated outputs. Statistical guarantees for the proposed method are studied and its advantageous features are demonstrated through numerical studies.