Classifier learning with a new locality regularization method
Classifier learning with a new locality regularization method
复制标题
使用新的局部正则化方法进行分类器学习
DOI：
10.1016/j.patcog.2007.09.016
复制
发表时间：
2008-05
影响因子：
8
通讯作者：
Chen, Songcan
中科院分区：
文献类型：
--
作者：
Zeng, Xiaoqin;Xue, Hui;Chen, Songcan
众所周知，泛化能力是开发和评价一个给定模式分类问题的分类器的最重要标准之一。Ng等人最近提出的局部化泛化误差模型(RSM)。[局部泛化误差及其在径向基函数神经网络训练中的应用，见：机器学习和控制论国际会议论文集，中国，2005；使用径向基函数神经网络进行图像分类和最小化局部泛化误差，模式识别40(1)(2007)4-18]提供了对泛化误差的更直观的观察。虽然RSM给出了一种全新的方法来提高泛化性能，但它在本质上等价于另一种类型的正则化。本文首先证明了RSM与正则化之间的本质联系，证明了RSM中的随机敏感度恰好对应于正则项。然后，我们从正则化的观点出发，给出了一个新的推广误差界，该误差界是从RSM和正则化之间证明的关系得到的。此外，我们还由此得到了一种新的正则化方法，称为局部性正则化(LR)。不同于现有的人工和外部附加正则化项以求光滑解的正则化方法，LR是从定义的期望风险泛函自然地和内部推导出来的，并利用局部信息来计算。该算法结合谱图理论，将样本的局部结构信息引入到正则化项中，进一步提高了泛化能力。与RSM对样本的不同采样相对敏感不同，LR在RSM中使用离散的k-邻域而不是常用的连续Q-邻域来自动区分不同训练样本的相对位置，避免了对不同分类器的复杂的Q计算。此外，LR使用正则化参数来控制训练精度和分类器稳定性之间的权衡。在人工和真实问题上的实验结果表明，LR比RSM和一些传统的正则化方法都具有更好的泛化能力。
It is well known that the generalization capability is one of the most important criterions to develop and evaluate a classifier for a given pattern classification problem. The localized generalization error model (RSM) recently proposed by Ng et al. [Localized generalization error and its application to RBFNN training, in: Proceedings of the International Conference on Machine Learning and Cybernetics, China, 2005; Image classification with the use of radial basis function neural networks and the minimization of the localized generalization error, Pattern Recognition 40(1) (2007) 4–18] provides a more intuitive look at the generalization error. Although RSMgives a brand-new method to promote the generalization performance, it is in nature equivalent to another type of regularization. In this paper, we first prove the essential relationship between RSMand regularization, and demonstrate that the stochastic sensitivity measure in RSMexactly corresponds to a regularizing term. Then, we develop a new generalization error bound from the regularization viewpoint, which is inspired by the proved relationship between RSMand regularization. Moreover, we derive a new regularization method, called as locality regularization (LR), from the bound. Different from the existing regularization methods which artificially and externally append the regularizing term in order to smooth the solution, LR is naturally and internally deduced from the defined expected risk functional and calculated by employing locality information. Through combining with spectral graph theory, LR introduces the local structure information of the samples into the regularizing term and further improves the generalization capability. In contrast with RSM, which is relatively sensitive to the different sampling of the samples, LR uses the discrete k-neighborhood rather than the common continuous Q-neighborhood in RSMto differentiate the relative position of different training samples automatically and avoid the complex computation of Q for various classifiers. Furthermore, LR uses the regularization parameter to control the trade-off between the training accuracy and the classifier stability. Experimental results on artificial and real world problems show that LR yields better generalization capability than both RSMand some traditional regularization methods.
登录
查看更多内容
DOI：
10.1017/cbo9780511801389.013
发表时间：
2000-03
期刊：
--
影响因子：
--
作者：
N. Cristianini;J. Shawe-Taylor
通讯作者：
N. Cristianini;J. Shawe-Taylor
DOI：
--
发表时间：
1997
期刊：
--
影响因子：
--
作者：
F. Chung
通讯作者：
F. Chung
DOI：
10.1007/978-1-4471-0285-4
发表时间：
2012-10
期刊：
--
影响因子：
--
作者：
Shigeo Abe DrEng
通讯作者：
Shigeo Abe DrEng
影响因子：
1.1
作者：
Wing W. Y. Ng;D. Yeung
通讯作者：
Wing W. Y. Ng;D. Yeung
影响因子：
2.5
作者：
R. Lordo
通讯作者：
R. Lordo