SVM-Boosting based on Markov resampling: Theory and algorithm
SVM-Boosting based on Markov resampling: Theory and algorithm
复制标题
基于马尔可夫重采样的SVM-Boosting：理论与算法
DOI：
10.1016/j.neunet.2020.07.036
复制
发表时间：
2020-11-01
期刊：
影响因子：
7.8
通讯作者：
Tang, Yuan Yan
中科院分区：
文献类型：
--
作者：
Jiang, Hongwei;Zou, Bin;Tang, Yuan Yan
在本文中，我们介绍了马尔可夫重采样的思想。首先证明了基于一致遍历马尔可夫链（u.e.M.c）的一般凸损失函数Boosting算法是一致的，并证明了其快速收敛速度。将基于马尔可夫重采样的Boosting算法应用于支持向量机（SVM），提出了两种新的基于重采样的Boosting算法：基于马尔可夫重采样的SVM-Boosting （SVM- bm）和基于马尔可夫重采样的改进SVM-Boosting （ISVM-BM）。与SVM-BM相比，ISVM-BM使用支持向量来计算基分类器的权重。基于基准数据集的数值研究表明，与三种经典AdaBoost算法（Gentle AdaBoost、Real AdaBoost、Modest AdaBoost）相比，本文提出的两种基于重采样的线性基分类器SVM Boosting算法具有更小的误分类率、更少的采样和训练总时间。此外，我们还将所提出的SVM-BM算法与目前广泛使用的高效梯度增强算法xgboost （eXtreme gradient Boosting）、SVM-AdaBoost进行了比较，并对技术参数进行了有益的讨论。(C) 2020 Elsevier Ltd.版权所有。
In this article we introduce the idea of Markov resampling for Boosting methods. We first prove that Boosting algorithm with general convex loss function based on uniformly ergodic Markov chain (u.e.M.c.) examples is consistent and establish its fast convergence rate. We apply Boosting algorithm based on Markov resampling to Support Vector Machine (SVM), and introduce two new resampling-based Boosting algorithms: SVM-Boosting based on Markov resampling (SVM-BM) and improved SVM-Boosting based on Markov resampling (ISVM-BM). In contrast with SVM-BM, ISVM-BM uses the support vectors to calculate the weights of base classifiers. The numerical studies based on benchmark datasets show that the proposed two resampling-based SVM Boosting algorithms for linear base classifiers have smaller misclassification rates, less total time of sampling and training compared to three classical AdaBoost algorithms: Gentle AdaBoost, Real AdaBoost, Modest AdaBoost. In addition, we compare the proposed SVM-BM algorithm with the widely used and efficient gradient Boosting algorithm-XGBoost (eXtreme Gradient Boosting), SVM-AdaBoost and present some useful discussions on the technical parameters. (C) 2020 Elsevier Ltd. All rights reserved.