Incremental Ensemble Gaussian Processes
Incremental Ensemble Gaussian Processes
复制标题
DOI：
10.1109/tpami.2022.3157197
复制
发表时间：
2021-10
影响因子：
23.6
通讯作者：
Qin Lu;G. V. Karanikolas;G. Giannakis
中科院分区：
文献类型：
--
作者：
Qin Lu;G. V. Karanikolas;G. Giannakis
高斯过程（Gaussian process，GP）方法属于贝叶斯非参数化方法，不仅在学习一类丰富的非线性函数方面具有良好的性能，而且在量化相关的不确定性方面也具有良好的性能。然而，大多数GP方法依赖于单个预选的核函数，这可能无法表征在时间关键型应用中顺序到达的数据样本。为了使在线内核自适应，目前的工作主张一个增量合奏（IE-）GP框架，其中EGP汇编器采用的GP学习者，每个人都有一个独特的内核属于一个规定的内核字典的合奏。随着每个GP专家利用基于随机特征的近似来执行具有可扩展性的在线预测和模型更新，EGP汇编器利用数据自适应权重来合成每个专家的预测。此外，新的IE-GP的广义适应时变函数建模结构化动态在EGP汇编器和每个GP学习器。为了在违反建模假设的对抗环境中对IE-GP及其动态变体的性能进行基准测试，通过后悔的概念进行了严格的性能分析，作为在线凸优化的规范。最后，在IE-GP框架下，研究了在线无监督学习降维方法。合成和真实的数据测试证明了所提出的方案的有效性。
Belonging to the family of Bayesian nonparametrics, Gaussian process (GP) based approaches have well-documented merits not only in learning over a rich class of nonlinear functions, but also in quantifying the associated uncertainty. However, most GP methods rely on a single preselected kernel function, which may fall short in characterizing data samples that arrive sequentially in time-critical applications. To enable online kernel adaptation, the present work advocates an incremental ensemble (IE-) GP framework, where an EGP assembler employs an ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary. With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with scalability, the EGP assembler capitalizes on data-adaptive weights to synthesize the per-expert predictions. Further, the novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP assembler and within each GP learner. To benchmark the performance of IE-GP and its dynamic variant in the adversarial setting where the modeling assumptions are violated, rigorous performance analysis has been conducted via the notion of regret, as the norm in online convex optimization. Last but not the least, online unsupervised learning for dimensionality reduction is explored under the novel IE-GP framework. Synthetic and real data tests demonstrate the effectiveness of the proposed schemes.