Probabilistic Neural-Kernel Tensor Decomposition
Probabilistic Neural-Kernel Tensor Decomposition
复制标题
DOI：
10.1109/icdm50108.2020.00062
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Conor Tillinghast;Shikai Fang;Kai Zhang;Shandian Zhe
中科院分区：
文献类型：
--
作者：
Conor Tillinghast;Shikai Fang;Kai Zhang;Shandian Zhe
张量分解是一个基本的框架来建模和分析多维数据，这是普遍存在于现实世界中的应用。张量分解的一个关键挑战是捕获各种复杂的关系/交互，同时避免过度拟合通常非常稀疏的数据。虽然已经提出了许多张量分解方法，但它们大多基于多线性形式，因此无法估计更复杂的非线性关系。为了应对这一挑战，我们提出了POND，PrObjective神经核张量分解，它将贝叶斯非参数函数学习的自适应和神经网络的表达能力结合起来。POND使用高斯过程（GP）来模拟隐藏的关系，并可以自动检测它们在张量中的复杂性，防止欠拟合和过拟合。然后，POND结合卷积神经网络来构建GP内核，以极大地提高估计高度非线性关系的能力。为了将POND扩展到大数据，我们使用稀疏变分GP框架和重新参数化技巧来开发一个有效的随机变分学习算法。在合成和真实世界的基准数据集上，POND通常表现出比最先进的非线性张量分解方法更好的预测性能。此外，作为贝叶斯方法，POND提供了潜在因素的后验分布，因此可以方便地量化其不确定性和预测的置信水平。
Tensor decomposition is a fundamental framework to model and analyze multiway data, which are ubiquitous in realworld applications. A critical challenge of tensor decomposition is to capture a variety of complex relationships/interactions while avoiding overfitting the data that are usually very sparse. Although numerous tensor decomposition methods have been proposed, they are mostly based on a multilinear form and hence are incapable of estimating more complex, nonlinear relationships. To address the challenge, we propose POND, PrObabilistic Neural-kernel tensor Decomposition that unifies the self-adaptation of Bayes nonparametric function learning and the expressive power of neural networks. POND uses Gaussian processes (GPs) to model the hidden relationships and can automatically detect their complexity in tensors, preventing both underfitting and overfitting. POND then incorporates convolutional neural networks to construct the GP kernel to greatly promote the capability of estimating highly nonlinear relationships. To scale POND to large data, we use the sparse variational GP framework and reparameterization trick to develop an efficient stochastic variational learning algorithm. On both synthetic and real-world benchmark datasets, POND often exhibits better predictive performance than the state-of-the-art nonlinear tensor decomposition methods. In addition, as a Bayesian approach, POND provides the posterior distribution of the latent factors, and hence can conveniently quantify their uncertainty and the confidence levels for predictions.