Envelopes and principal component regression
Envelopes and principal component regression
复制标题
DOI：
10.1214/23-ejs2154
复制
发表时间：
2022-07
影响因子：
1.1
通讯作者：
Xin Zhang;Kai Deng;Qing Mai
中科院分区：
文献类型：
--
作者：
Xin Zhang;Kai Deng;Qing Mai
包络方法为各种模型提供了有针对性的降维。首要目标是通过将数据投影到称为包络的低维子空间来提高多变量参数估计的效率。包络方法在分析变量高度相关的数据方面具有优势，但其迭代Grassman优化算法不能很好地扩展到超高维数据。虽然多元线性回归中包络与偏最小二乘之间的联系促进了包络高维研究的最新进展，但我们从一个新的主成分回归的角度提出了一种更直接的包络建模方法。本文提出的非迭代包络分量估计方法(NICEIE)比高维迭代Grassman优化方法具有更好的计算优势。我们发展了一个统一的侄女理论，它弥合了包络方法和回归中的主成分之间的差距。这些新的理论见解还揭示了包络子空间估计误差作为用于包络建模的两个对称正定矩阵的特征值间隙的函数。我们将新的理论和算法应用于几个包络模型，包括多元线性模型中的响应和预测因子缩减、Logistic回归和Cox比例风险模型。模拟和说明性数据分析表明，NICESS具有显著改进线性和广义线性模型中的标准方法的潜力。
Envelope methods offer targeted dimension reduction for various models. The overarching goal is to improve efficiency in multivariate parameter estimation by projecting the data onto a lower-dimensional subspace known as the envelope. Envelope approaches have advantages in analyzing data with highly correlated variables, but their iterative Grassmannian optimization algorithms do not scale very well with ultra high-dimensional data. While the connections between envelopes and partial least squares in multivariate linear regression have promoted recent progress in high-dimensional studies of envelopes, we propose a more straightforward way of envelope modeling from a novel principal components regression perspective. The proposed procedure, Non-Iterative Envelope Component Estimation (NIECE), has excellent computational advantages over the iterative Grassmannian optimization alternatives in high dimensions. We develop a unified NIECE theory that bridges the gap between envelope methods and principal components in regression. The new theoretical insights also shed light on the envelope subspace estimation error as a function of eigenvalue gaps of two symmetric positive definite matrices used in envelope modeling. We apply the new theory and algorithm to several envelope models, including response and predictor reduction in multivariate linear models, logistic regression, and Cox proportional hazard model. Simulations and illustrative data analysis show the potential for NIECE to improve standard methods in linear and generalized linear models significantly.