Ridge Regression: Structure, Cross-Validation, and Sketching
Ridge Regression: Structure, Cross-Validation, and Sketching
复制标题
DOI：
--
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Sifan Liu;Edgar Dobriban
中科院分区：
文献类型：
--
作者：
Sifan Liu;Edgar Dobriban
本文研究岭回归的三个基本问题：（1）估计量的结构是什么？(2)如何正确使用交叉验证来选择正则化参数？以及（3）如何在不损失太多准确性的情况下加速计算？我们考虑在一个统一的大数据线性模型的三个问题。我们给出了一个精确的表示岭回归作为一个协方差矩阵依赖的线性组合的真实参数和噪声。本文研究了K折交叉验证中正则化参数选择的偏差，并提出了一种简单的偏差修正方法。我们分析了岭回归的原始和对偶草图的准确性，表明它们是惊人的准确。我们的结果说明了模拟和分析经验数据。
We study the following three fundamental problems about ridge regression: (1) what is the structure of the estimator? (2) how to correctly use cross-validation to choose the regularization parameter? and (3) how to accelerate computation without losing too much accuracy? We consider the three problems in a unified large-data linear model. We give a precise representation of ridge regression as a covariance matrix-dependent linear combination of the true parameter and the noise. We study the bias of $K$-fold cross-validation for choosing the regularization parameter, and propose a simple bias-correction. We analyze the accuracy of primal and dual sketching for ridge regression, showing they are surprisingly accurate. Our results are illustrated by simulations and by analyzing empirical data.