Learning Rates of lq Coefficient Regularization Learning with Gaussian Kernel
Learning Rates of lq Coefficient Regularization Learning with Gaussian Kernel
复制标题
DOI：
10.1162/neco_a_00641
复制
发表时间：
2013-12
影响因子：
2.9
通讯作者：
Shaobo Lin;Jinshan Zeng;Jian Fang;Zongben Xu
中科院分区：
文献类型：
--
作者：
Shaobo Lin;Jinshan Zeng;Jian Fang;Zongben Xu
正则化是一种公认的提高学习机性能的强大策略，lq正则化方案是使用的中心。众所周知，不同的q会导致推导出的估计量的不同性质，例如，l2正则化会导致光滑估计量，而l1正则化会导致稀疏估计量。那么lq正则化学习的泛化能力随q的变化是值得研究的。在这封信中，我们研究这个问题的统计学习理论的框架。我们的主要结果表明，实施lq系数正则化方案的样本依赖的假设空间与高斯核可以达到几乎相同的最佳学习率。也就是说，lq正则化学习的学习率的上界和下界对于所有都是渐近相同的。我们的研究结果初步表明，在某些建模环境中，q的选择可能不会对泛化能力产生很大影响。从这个角度来看，q可以被任意指定，或者仅仅由其他非泛化标准（如平滑性，计算复杂性或稀疏性）指定。
Regularization is a well-recognized powerful strategy to improve the performance of a learning machine and lq regularization schemes with are central in use. It is known that different q leads to different properties of the deduced estimators, say, l2 regularization leads to a smooth estimator, while l1 regularization leads to a sparse estimator. Then how the generalization capability of lq regularization learning varies with q is worthy of investigation. In this letter, we study this problem in the framework of statistical learning theory. Our main results show that implementing lq coefficient regularization schemes in the sample-dependent hypothesis space associated with a gaussian kernel can attain the same almost optimal learning rates for all . That is, the upper and lower bounds of learning rates for lq regularization learning are asymptotically identical for all . Our finding tentatively reveals that in some modeling contexts, the choice of q might not have a strong impact on the generalization capability. From this perspective, q can be arbitrarily specified, or specified merely by other nongeneralization criteria like smoothness, computational complexity or sparsity.