Tensor Regression Using Low-rank and Sparse Tucker Decompositions
Tensor Regression Using Low-rank and Sparse Tucker Decompositions
复制标题
DOI：
10.1137/19m1299335
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Talal Ahmed;Haroon Raja;W. Bajwa
中科院分区：
文献类型：
--
作者：
Talal Ahmed;Haroon Raja;W. Bajwa
本文研究了一种带有标量响应变量和张量结构预测变量的张量结构线性回归模型，使得回归参数在$\mathbb{R}^{n_1 \times n_2 \times \cdots \times n_d}$中形成$d$阶张量（即$d$折叠多路数组）。它专注于根据响应变量和预测变量的 $m$ 实现来估计回归张量的任务，其中 $m\ll n = \prod \nolimits_{i} n_i$。尽管这个问题看起来不适定，但如果参数张量属于稀疏、低塔克秩张量的空间，它仍然可以解决。因此，估计过程被视为稀疏、低塔克秩张量空间上的非凸优化程序，并且提出了投影梯度下降的张量变体来解决由此产生的非凸问题。此外，还提供了数学保证，使所提出的方法在一组特定条件下线性收敛到适当的解决方案。此外，当各个（标量）预测器独立地从亚高斯分布中提取值时，所考虑的模型的张量参数估计的样本复杂性的上限被表征为特殊情况。样本复杂度界限显示出对 $\bar{n} = \max \big\{n_i: i\in \{1,2,\ldots,d \} \big\}$ 具有多对数依赖性，并且按顺序，它与可以从启发式参数计数参数获得的界限相匹配。最后，数值实验证明了所提出的张量模型和估计方法在与注意缺陷多动障碍相关的合成数据集和神经影像数据集集合上的有效性。具体来说，所提出的方法在合成数据集和真实数据集上都表现出更好的样本复杂性，证明了模型和方法在 $n \gg m$ 的设置中的有用性。
This paper studies a tensor-structured linear regression model with a scalar response variable and tensor-structured predictors, such that the regression parameters form a tensor of order $d$ (i.e., a $d$-fold multiway array) in $\mathbb{R}^{n_1 \times n_2 \times \cdots \times n_d}$. It focuses on the task of estimating the regression tensor from $m$ realizations of the response variable and the predictors where $m\ll n = \prod \nolimits_{i} n_i$. Despite the seeming ill-posedness of this problem, it can still be solved if the parameter tensor belongs to the space of sparse, low Tucker-rank tensors. Accordingly, the estimation procedure is posed as a non-convex optimization program over the space of sparse, low Tucker-rank tensors, and a tensor variant of projected gradient descent is proposed to solve the resulting non-convex problem. In addition, mathematical guarantees are provided that establish the proposed method linearly converges to an appropriate solution under a certain set of conditions. Further, an upper bound on sample complexity of tensor parameter estimation for the model under consideration is characterized for the special case when the individual (scalar) predictors independently draw values from a sub-Gaussian distribution. The sample complexity bound is shown to have a polylogarithmic dependence on $\bar{n} = \max \big\{n_i: i\in \{1,2,\ldots,d \} \big\}$ and, orderwise, it matches the bound one can obtain from a heuristic parameter counting argument. Finally, numerical experiments demonstrate the efficacy of the proposed tensor model and estimation method on a synthetic dataset and a collection of neuroimaging datasets pertaining to attention deficit hyperactivity disorder. Specifically, the proposed method exhibits better sample complexities on both synthetic and real datasets, demonstrating the usefulness of the model and the method in settings where $n \gg m$.