Fractional ridge regression: a fast, interpretable reparameterization of ridge regression.
Fractional ridge regression: a fast, interpretable reparameterization of ridge regression.
复制标题
DOI：
10.1093/gigascience/giaa133
复制
发表时间：
2020-11-30
期刊：
影响因子：
9.2
通讯作者：
Kay K
中科院分区：
文献类型：
--
作者：
Rokem A;Kay K
岭回归是一种正则化技术，它惩罚线性回归中系数的L2范数。使用岭回归的挑战之一是需要设置一个控制正则化量的超参数（α）。交叉验证通常用于从一组候选者中选择最佳α。然而，有效和适当地选择α可能具有挑战性。在分析大量数据时，这变得非常困难。由于选定的α取决于数据的规模和预测变量之间的相关性，因此也无法直接解释。目前的工作通过一种新的岭回归方法来解决这些挑战。我们建议重新参数化岭回归的正则化和非正则化系数的L2-范数之间的比率γ。我们提供了一种有效实现这种方法的算法，称为分数岭回归，以及Python和matlab中的开源软件实现（https：//github.com/nrdg/fracridge）。我们表明，该方法是快速和可扩展的大规模数据问题。在脑成像数据中，我们证明了这种方法提供的结果是直接解释和比较的模型和数据集。分数岭回归有几个好处：保证不同γ下得到的解是不同的，避免了浪费计算;自动跨越正则化的相关范围，避免了繁重的人工探索。这些性质使得分数岭回归特别适合于分析大型复杂数据集。
Ridge regression is a regularization technique that penalizes the L2-norm of the coefficients in linear regression. One of the challenges of using ridge regression is the need to set a hyperparameter (α) that controls the amount of regularization. Cross-validation is typically used to select the best α from a set of candidates. However, efficient and appropriate selection of α can be challenging. This becomes prohibitive when large amounts of data are analyzed. Because the selected α depends on the scale of the data and correlations across predictors, it is also not straightforwardly interpretable. The present work addresses these challenges through a novel approach to ridge regression. We propose to reparameterize ridge regression in terms of the ratio γ between the L2-norms of the regularized and unregularized coefficients. We provide an algorithm that efficiently implements this approach, called fractional ridge regression, as well as open-source software implementations in Python and matlab (https://github.com/nrdg/fracridge). We show that the proposed method is fast and scalable for large-scale data problems. In brain imaging data, we demonstrate that this approach delivers results that are straightforward to interpret and compare across models and datasets. Fractional ridge regression has several benefits: the solutions obtained for different γ are guaranteed to vary, guarding against wasted calculations; and automatically span the relevant range of regularization, avoiding the need for arduous manual exploration. These properties make fractional ridge regression particularly suitable for analysis of large complex datasets.
登录
查看更多内容
影响因子：
64.8
作者：
Harris CR;Millman KJ;van der Walt SJ;Gommers R;Virtanen P;Cournapeau D;Wieser E;Taylor J;Berg S;Smith NJ;Kern R;Picus M;Hoyer S;van Kerkwijk MH;Brett M;Haldane A;Del Río JF;Wiebe M;Peterson P;Gérard-Marchant P;Sheppard K;Reddy T;Weckesser W;Abbasi H;Gohlke C;Oliphant TE
通讯作者：
Oliphant TE
影响因子：
2.5
作者：
HOERL, AE;KENNARD, RW
通讯作者：
KENNARD, RW
影响因子：
2.5
作者：
GOLUB, GH;HEATH, M;WAHBA, G
通讯作者：
WAHBA, G
DOI：
10.1073/pnas.1900654116
发表时间：
2019-10-29
影响因子：
11.1
作者：
Murdoch, W. James;Singh, Chandan;Yu, Bin
通讯作者：
Yu, Bin
影响因子：
2.2
作者：
SKOURAS, K;GOUTIS, C;BRAMSON, MJ
通讯作者：
BRAMSON, MJ