The Generalization Performance of Regularized Regression Algorithms Based on Markov Sampling
The Generalization Performance of Regularized Regression Algorithms Based on Markov Sampling
复制标题
DOI：
10.1109/tcyb.2013.2287191
复制
发表时间：
2014-09
影响因子：
11.8
通讯作者：
Bin Zou;Yuanyan Tang;Zongben Xu;Luoqing Li;Jie Xu;Yang Lu
中科院分区：
文献类型：
--
作者：
Bin Zou;Yuanyan Tang;Zongben Xu;Luoqing Li;Jie Xu;Yang Lu
本文研究了两种基于非独立同分布（non-i.i.d.）的正则化回归算法[最小二乘正则化回归（LSRR）和支持向量机回归（SVMR）]的泛化能力。样品不同于先前已知的用于非i.i.d.本文研究了基于均匀遍历马尔可夫链（u.e.M.c.）样品从马尔可夫链蒙特卡罗（MCMC）方法的想法的启发，我们还介绍了一个新的马尔可夫抽样算法的回归生成u.e.M.c.样本，然后，我们提出了基于马尔可夫抽样的LSRR和SVMR的学习性能的数值研究。实验结果表明，与随机抽样相比，基于马尔可夫抽样的LSRR和SVMR具有更小的均方误差和方差。
This paper considers the generalization ability of two regularized regression algorithms [least square regularized regression (LSRR) and support vector machine regression (SVMR)] based on non-independent and identically distributed (non-i.i.d.) samples. Different from the previously known works for non-i.i.d. samples, in this paper, we research the generalization bounds of two regularized regression algorithms based on uniformly ergodic Markov chain (u.e.M.c.) samples. Inspired by the idea from Markov chain Monto Carlo (MCMC) methods, we also introduce a new Markov sampling algorithm for regression to generate u.e.M.c. samples from a given dataset, and then, we present the numerical studies on the learning performance of LSRR and SVMR based on Markov sampling, respectively. The experimental results show that LSRR and SVMR based on Markov sampling can present obviously smaller mean square errors and smaller variances compared to random sampling.