Generalization Bounds of Regularization Algorithms Derived Simultaneously through Hypothesis Space Complexity, Algorithmic Stability and Data Quality
Generalization Bounds of Regularization Algorithms Derived Simultaneously through Hypothesis Space Complexity, Algorithmic Stability and Data Quality
复制标题
DOI：
10.1142/s0219691311004213
复制
发表时间：
2011-11
期刊：
影响因子：
--
通讯作者：
Xiangyu Chang;Zongben Xu;Bin Zou;Hai Zhang
中科院分区：
文献类型：
--
作者：
Xiangyu Chang;Zongben Xu;Bin Zou;Hai Zhang
机器学习研究的一个主要问题是分析学习机的泛化性能。关于正则化算法推广性能的经典结果，大多数都是仅仅从假设空间的复杂性或学习算法的稳定性出发得到的。然而，在实际应用中，学习算法的性能实际上并不仅仅受到假设空间复杂度、算法稳定性和数据质量等单一因素的影响。因此，在本文中，我们开发了一个框架，评估正则化算法的泛化性能的假设空间复杂性，算法的稳定性和数据质量相结合。基于一致稳定性测度和经验覆盖数，对一般损失函数建立了正则化算法学习率的新界。作为一般结果的应用，我们评估了支持向量机的学习率。
A main issue in machine learning research is to analyze the generalization performance of a learning machine. Most classical results on the generalization performance of regularization algorithms are derived merely with the complexity of hypothesis space or the stability property of a learning algorithm. However, in practical applications, the performance of a learning algorithm is not actually affected only by an unitary factor just like the complexity of hypothesis space, stability of the algorithm and data quality. Therefore, in this paper, we develop a framework of evaluating the generalization performance of regularization algorithms combinatively in terms of hypothesis space complexity, algorithmic stability and data quality. We establish new bounds on the learning rate of regularization algorithms based on the measure of uniform stability and empirical covering number for general type of loss functions. As applications of the generic results, we evaluate the learning rates of support vector machine...