Stochastic Optimization for Coupled Tensor Decomposition with Applications in Statistical Learning
Stochastic Optimization for Coupled Tensor Decomposition with Applications in Statistical Learning
复制标题
DOI：
10.1109/dsw.2019.8755797
复制
发表时间：
2019-06
期刊：
影响因子：
--
通讯作者：
Shahana Ibrahim;Xiao Fu
中科院分区：
文献类型：
--
作者：
Shahana Ibrahim;Xiao Fu
耦合张量分解的目的是分解一些共享其潜在因子的张量。现有的耦合正则多元分解(CPD)算法面临着严重的可伸缩性挑战，特别是在张量数目很大的情况下。然而，在诸如统计学习之类的及时应用中，例如当从边际概率质量函数估计许多随机变量的联合概率质量函数(PMF)时，自然会出现大量的耦合张量。对于耦合分解，存在允许轻量级更新的随机算法，但由于其采样模式，这些算法不能处理复杂约束(例如，在统计学习中重要的概率单纯形约束)。这项工作提出了一种简单的数据采样和块变量更新策略，可以同时分解大量耦合张量。该算法具有较低的单次迭代复杂度，并能很容易地处理潜在因素的约束。我们还表明，这种多块算法与经典的单块随机邻近梯度(SPG)有很好的联系，因此它自然地继承了SPG的收敛性质。综合实验和真实数据实验表明，该算法在统计学习问题上具有很好的应用前景。
Coupled tensor decomposition aims at factoring a number of tensors that share some of their latent factors. Existing algorithms for coupled canonical polyadic decomposition (CPD) face serious scal-ablity challenges, especially when the number of tensors is large. However, a large amount of coupled tensors naturally arise in timely applications such as statistical learning, e.g., when estimating the joint probability mass function (PMF) of many random variables from marginal PMFs. Stochastic algorithms that admit lightweight updates exist for coupled decomposition, but these algorithms cannot handle complex constraints (e.g., the probability simplex constraint that is important in statistical learning) due to their sampling patterns. This work puts forth a simple data-sampling and block variable-updating strategy for simultaneously factoring a large number of coupled tensors. The proposed algorithm enjoys low per-iteration complexity and can easily handle constraints on latent factors. We also show that this multi-block algorithm admits a nice connection to the classic single-block stochastic proximal gradient (SPG), and thus it naturally inherits convergence properties of SPG. Synthetic and real-data experiments show that the proposed algorithm is very promising for statistical learning problems.