Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes
Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes
复制标题
DOI：
--
复制
发表时间：
2007-12
期刊：
影响因子：
--
通讯作者：
R. Salakhutdinov;Geoffrey E. Hinton
中科院分区：
文献类型：
--
作者：
R. Salakhutdinov;Geoffrey E. Hinton
我们展示了如何使用未标记数据和深度信念网（DBN）来学习高斯过程的良好协方差核。我们首先使用[7]介绍的快速贪婪算法学习未标记数据的深度生成模型。如果数据是高维和高度结构化的，则应用于DBN中的顶层特征的高斯内核比应用于原始输入的类似内核要好得多。然后，通过使用DBN的反向传播来有区别地微调协方差核，可以进一步提高回归和分类的性能。
We show how to use unlabeled data and a deep belief net (DBN) to learn a good covariance kernel for a Gaussian process. We first learn a deep generative model of the unlabeled data using the fast, greedy algorithm introduced by [7]. If the data is high-dimensional and highly-structured, a Gaussian kernel applied to the top layer of features in the DBN works much better than a similar kernel applied to the raw input. Performance at both regression and classification can then be further improved by using backpropagation through the DBN to discriminatively fine-tune the covariance kernel.