On an improvement of LASSO by scaling
On an improvement of LASSO by scaling
复制标题
DOI：
--
复制
发表时间：
2018-08
期刊：
影响因子：
--
通讯作者：
K. Hagiwara
中科院分区：
文献类型：
--
作者：
K. Hagiwara
稀疏建模是机器学习和统计学中的一个主要课题。LASSO（Least Absolute Shrinkage and Selection Operator）是一种常用的稀疏建模方法，但它在稀疏表示时会产生意想不到的大偏差。已经有几个研究，以改善这个问题，如非凸正则化项的引入。重要的一点是，这种偏差问题直接影响应用程序中的模型选择，因为稀疏表示无法通过基于预测误差的模型选择来选择，即使它是一个很好的表示。在这篇文章中，我们考虑改善这个问题，通过引入一个扩展的LASSO估计，以补偿过度收缩，从而在LASSO估计的大偏差。我们在这里给出了缩放量的经验值。这种缩放方法有如下两个优点。由于所提出的尺度值是通过使用LASSO估计器来计算的，因此我们只需要通过快速且稳定的优化过程（例如LARS（最小角度回归））在LASSO修改或坐标下降下获得的LASSO估计器。而且，我们的缩放方法的简单性，使我们能够得到SURE（斯坦的无偏风险估计）下的修改LASSO估计与缩放。我们的缩放方法以及基于SURE的模型选择是完全经验的，不需要额外的超参数。在一个简单的数值例子中，我们验证了我们的缩放方法实际上改善了LASSO和基于SURE的模型选择标准可以稳定地选择合适的稀疏模型。
A sparse modeling is a major topic in machine learning and statistics. LASSO (Least Absolute Shrinkage and Selection Operator) is a popular sparse modeling method while it has been known to yield unexpected large bias especially at a sparse representation. There have been several studies for improving this problem such as the introduction of non-convex regularization terms. The important point is that this bias problem directly affects model selection in applications since a sparse representation cannot be selected by a prediction error based model selection even if it is a good representation. In this article, we considered to improve this problem by introducing a scaling that expands LASSO estimator to compensate excessive shrinkage, thus a large bias in LASSO estimator. We here gave an empirical value for the amount of scaling. There are two advantages of this scaling method as follows. Since the proposed scaling value is calculated by using LASSO estimator, we only need LASSO estimator that is obtained by a fast and stable optimization procedure such as LARS (Least Angle Regression) under LASSO modification or coordinate descent. And, the simplicity of our scaling method enables us to derive SURE (Stein's Unbiased Risk Estimate) under the modified LASSO estimator with scaling. Our scaling method together with model selection based on SURE is fully empirical and do not need additional hyper-parameters. In a simple numerical example, we verified that our scaling method actually improves LASSO and the SURE based model selection criterion can stably choose an appropriate sparse model.