A Scaling and Non-Negative Garrote in Soft-Thresholding
A Scaling and Non-Negative Garrote in Soft-Thresholding
复制标题
软阈值中的缩放和非负绞锁
DOI：
10.1587/transinf.2016edp7365
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
K. Hagiwara
中科院分区：
文献类型：
--
作者：
K. Hagiwara
软阈值是一种稀疏建模方法，通常应用于统计信号处理中的小波去噪。它在机器学习中也很重要，因为它是众所周知的套索(最小绝对收缩和选择算子)的基本性质。众所周知，软阈值方法面临着稀疏性和泛化之间的两难问题。这是由于稀疏表示的过度收缩造成的。在信号处理和机器学习领域，有几种方法可以改善这个问题。在本文中，我们考虑推广和分析一种软阈值估计量的尺度方法。在一个包含离散小波变换的非参数正交回归问题中，我们引入了分量和数据相关的尺度，它实际上与非负绞索是相同的。这里我们考虑了一种从最小二乘估计的绝对值中选择软阈值的参数值的情况，由此将模型选择问题归结为确定非零系数估计的个数。在这种情况下，我们首先导出了一种风险，并构造了可用于确定非零系数估计个数的SURE(Stein的无偏风险估计)。我们还分析了风险曲线的一些性质，发现我们的Scaling方法与简单的软阈值方法相比，可能产生低风险和高稀疏性的模型。通过一个简单的小波去噪数值实验，验证了这一理论推测。关键词：软阈值、SURE、非负绞线、尺度、小波去噪
Soft-thresholding is a sparse modeling method typically applied to wavelet denoising in statistical signal processing. It is also important in machine learning since it is an essential nature of the well-known LASSO (Least Absolute Shrinkage and Selection Operator). It is known that soft-thresholding, thus, LASSO suffers from a problem of dilemma between sparsity and generalization. This is caused by excessive shrinkage at a sparse representation. There are several methods for improving this problem in the field of signal processing and machine learning. In this paper, we considered to extend and analyze a method of scaling of soft-thresholding estimators. In a setting of non-parametric orthogonal regression problem including discrete wavelet transform, we introduced component-wise and data-dependent scaling that is indeed identical to non-negative garrote. We here considered a case where a parameter value of soft-thresholding is chosen from absolute values of the least squares estimates, by which the model selection problem reduces to the determination of the number of non-zero coefficient estimates. In this case, we firstly derived a risk and construct SURE (Stein’s unbiased risk estimator) that can be used for determining the number of non-zero coefficient estimates. We also analyzed some properties of the risk curve and found that our scaling method with the derived SURE is possible to yield a model with low risk and high sparsity compared to a naive soft-thresholding method with SURE. This theoretical speculation was verified by a simple numerical experiment of wavelet denoising. key words: soft-thresholding, SURE, non-negative garrote, scaling, wavelet denoising
具有可变分量的正交回归的预期预测误差的估计
DOI：
--
发表时间：
2007
期刊：
影响因子：
--
作者：
Katsuyuki Hagiwara;Hiroshi Ishitani
通讯作者：
Hiroshi Ishitani
DOI：
10.1198/016214506000000735
发表时间：
2006-12-01
影响因子：
3.7
作者：
Zou, Hui
通讯作者：
Zou, Hui