HIGH-DIMENSIONAL ASYMPTOTICS OF PREDICTION: RIDGE REGRESSION AND CLASSIFICATION
HIGH-DIMENSIONAL ASYMPTOTICS OF PREDICTION: RIDGE REGRESSION AND CLASSIFICATION
复制标题
DOI：
10.1214/17-aos1549
复制
发表时间：
2018-02-01
影响因子：
4.5
通讯作者：
Wager, Stefan
中科院分区：
文献类型：
--
作者：
Dobriban, Edgar;Wager, Stefan
我们提供了一个统一的分析岭回归和正则化判别分析在密集随机效应模型的预测风险。我们在高维渐近状态下工作，其中p，n ->无穷大和p/n -> gamma > 0，并允许特征之间的任意协方差。对于这两种方法，我们提供了一个明确的和有效的计算表达式的限制预测风险，这只取决于频谱的特征协方差矩阵，信号强度和纵横比。特别是在正则化判别分析的情况下，我们发现，预测精度有一个微妙的依赖于协方差矩阵的特征值分布，这表明基于协方差矩阵的算子范数的分析可能不尖锐。我们的研究结果还揭示了高维线性模型中极限预测风险和极限估计风险之间的精确逆关系。该分析建立在随机矩阵理论的最新进展之上。
We provide a unified analysis of the predictive risk of ridge regression and regularized discriminant analysis in a dense random effects model. We work in a high-dimensional asymptotic regime where p, n -> infinity and p/n -> gamma > 0, and allow for arbitrary covariance among the features. For both methods, we provide an explicit and efficiently computable expression for the limiting predictive risk, which depends only on the spectrum of the feature-covariance matrix, the signal strength and the aspect ratio.. Especially in the case of regularized discriminant analysis, we find that predictive accuracy has a nuanced dependence on the eigenvalue distribution of the covariance matrix, suggesting that analyses based on the operator norm of the covariance matrix may not be sharp. Our results also uncover an exact inverse relation between the limiting predictive risk and the limiting estimation risk in high-dimensional linear models. The analysis builds on recent advances in random matrix theory.