A study on tuning parameter selection for the high-dimensional lasso
A study on tuning parameter selection for the high-dimensional lasso
复制标题
DOI：
10.1080/00949655.2018.1491575
复制
发表时间：
2016-02
影响因子：
1.2
通讯作者：
D. Homrighausen;D. McDonald
中科院分区：
文献类型：
--
作者：
D. Homrighausen;D. McDonald
摘要高维预测模型，即测量值多于观测值的模型，需要正则化来定义好，经验上表现良好，并具有理论上的保证。正则化的数量通常由调整参数决定，对于实现良好的性能至关重要。人们可以通过多种方式选择调优参数，例如通过响应方法或广义信息准则。然而，支持许多正则化过程的理论依赖于对方差参数的估计，这在高维中是复杂的。我们开发了一套信息标准选择调整参数的套索回归利用文献的高维方差估计。我们得到的直觉表明，现有的信息理论的方法在这种情况下工作不佳。我们比较我们的风险估计现有的方法与广泛的模拟，并得出一些理论依据。我们发现，我们的新的估计在广泛的模拟条件和评价标准表现良好。
ABSTRACT High-dimensional predictive models, those with more measurements than observations, require regularization to be well defined, perform well empirically, and possess theoretical guarantees. The amount of regularization, often determined by tuning parameters, is integral to achieving good performance. One can choose the tuning parameter in a variety of ways, such as through resampling methods or generalized information criteria. However, the theory supporting many regularized procedures relies on an estimate for the variance parameter, which is complicated in high dimensions. We develop a suite of information criteria for choosing the tuning parameter in lasso regression by leveraging the literature on high-dimensional variance estimation. We derive intuition showing that existing information-theoretic approaches work poorly in this setting. We compare our risk estimators to existing methods with an extensive simulation and derive some theoretical justification. We find that our new estimators perform well across a wide range of simulation conditions and evaluation criteria.