Cross validation in LASSO and its acceleration
Cross validation in LASSO and its acceleration
复制标题
DOI：
10.1088/1742-5468/2016/05/053304
复制
发表时间：
2016-05-01
影响因子：
2.4
通讯作者：
Kabashima, Yoshiyuki
中科院分区：
文献类型：
--
作者：
Obuchi, Tomoyuki;Kabashima, Yoshiyuki
我们研究留一交叉验证（CV）作为最小绝对收缩和选择算子（LASSO）中惩罚项权重的决定因素。首先，基于消息传递算法和微扰讨论假设观测的数量是足够大的，我们提供了简单的公式近似评估两种类型的CV错误，这使我们能够显着降低必要的计算成本。这些公式还提供了一个简单的连接的CV误差的残差平方和之间的重建和给定的测量。其次，在此基础上，我们分析了CV误差时，设计矩阵是一个简单的随机矩阵，在大尺寸的限制，通过使用复制方法。最后，将所得结果与有限尺寸系统的数值模拟结果进行了比较，证实了其正确性。我们还将第一类CV误差的简单公式应用于超新星的实际数据集。
We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator (LASSO). First, on the basis of the message passing algorithm and a perturbative discussion assuming that the number of observations is sufficiently large, we provide simple formulas for approximately assessing two types of CV errors, which enable us to significantly reduce the necessary cost of computation. These formulas also provide a simple connection of the CV errors to the residual sums of squares between the reconstructed and the given measurements. Second, on the basis of this finding, we analytically evaluate the CV errors when the design matrix is given as a simple random matrix in the large size limit by using the replica method. Finally, these results are compared with those of numerical simulations on finite-size systems and are confirmed to be correct. We also apply the simple formulas of the first type of CV error to an actual dataset of the supernovae.