Structured Low-Rank Tensors for Generalized Linear Models
Structured Low-Rank Tensors for Generalized Linear Models
复制标题
DOI：
10.48550/arxiv.2308.02922
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
Batoul Taki;A. Sarwate;W. Bajwa
中科院分区：
文献类型：
--
作者：
Batoul Taki;A. Sarwate;W. Bajwa
最近的工作表明，在回归问题中对系数张量施加张量结构可以导致更可靠的参数估计和更低的样本复杂度。本文研究了广义线性模型（GLM）问题中一种新的低秩张量模型，称为低分离秩（LSR）。LSR模型-它概括了著名的Tucker和CANDECOMP/PARAFAC（CP）模型，是块张量分解（BTD）模型的特殊情况-被施加到GLM模型中的系数张量上。本文提出了一种用于LR结构张量GLM参数估计的块坐标下降算法。最重要的是，它得到了一个极小极大的误差阈值估计的LSR张量GLM问题的系数张量的下限。极大极小界与LSR张量GLM问题中的固有自由度成比例，表明其样本复杂度可能显著低于向量化GLM。该结果也可以专门用于降低CP和Tucker结构GLM中的估计误差。推导出的界限是可比的紧界在文献中的塔克线性回归，并进一步评估数值的最小最大下限的紧密性。最后，在人工数据集上的数值实验证明了所提出的LSR张量模型对三种回归类型（线性，逻辑和泊松）的有效性。一组医学成像数据集上的实验证明了LSR模型在有限样本的真实的不平衡数据上优于其他张量模型（Tucker和CP）的有用性。
Recent works have shown that imposing tensor structures on the coefficient tensor in regression problems can lead to more reliable parameter estimation and lower sample complexity compared to vector-based methods. This work investigates a new low-rank tensor model, called Low Separation Rank (LSR), in Generalized Linear Model (GLM) problems. The LSR model -- which generalizes the well-known Tucker and CANDECOMP/PARAFAC (CP) models, and is a special case of the Block Tensor Decomposition (BTD) model -- is imposed onto the coefficient tensor in the GLM model. This work proposes a block coordinate descent algorithm for parameter estimation in LSR-structured tensor GLMs. Most importantly, it derives a minimax lower bound on the error threshold on estimating the coefficient tensor in LSR tensor GLM problems. The minimax bound is proportional to the intrinsic degrees of freedom in the LSR tensor GLM problem, suggesting that its sample complexity may be significantly lower than that of vectorized GLMs. This result can also be specialised to lower bound the estimation error in CP and Tucker-structured GLMs. The derived bounds are comparable to tight bounds in the literature for Tucker linear regression, and the tightness of the minimax lower bound is further assessed numerically. Finally, numerical experiments on synthetic datasets demonstrate the efficacy of the proposed LSR tensor model for three regression types (linear, logistic and Poisson). Experiments on a collection of medical imaging datasets demonstrate the usefulness of the LSR model over other tensor models (Tucker and CP) on real, imbalanced data with limited available samples.