Decision Tree SVM: An extension of linear SVM for non-linear classification
Decision Tree SVM: An extension of linear SVM for non-linear classification
复制标题
DOI：
10.1016/j.neucom.2019.10.051
复制
发表时间：
2020-08
期刊：
影响因子：
6
通讯作者：
F. Nie;Wei Zhu;Xuelong Li
中科院分区：
文献类型：
--
作者：
F. Nie;Wei Zhu;Xuelong Li
核技巧被广泛应用于支持向量机（SVM）中来处理线性不可分的数据，称为核SVM。然而，核支持向量机在实际应用中存在计算量大的问题，不适合处理大规模数据。此外，核支持向量机总是带来超参数，例如高斯核的带宽。由于超参数对核支持向量机的最终性能有很大的影响，并且很难调整，特别是对于大规模数据，人们可能需要花费大量的精力来寻找足够好的参数，并且超参数的不适当设置往往会使分类性能甚至低于线性支持向量机。受线性支持向量机处理大规模数据的最新进展的启发，我们提出了一个设计良好的分类器来有效地处理大规模线性不可分的数据，即，决策树支持向量机（DTSVM）。与核支持向量机相比，DTSVM具有更低的计算成本，并且除了在实践中可以固定的几个阈值外，几乎不引入超参数。在大规模数据集上的综合实验证明了该方法的优越性。
Kernel trick is widely applied to Support Vector Machine (SVM) to deal with linearly inseparable data which is known as kernel SVM. However, kernel SVM always has high computational cost in practice which makes it unsuitable to handle large scale data. Moreover, kernel SVM always brings hyper-parameters, e.g. bandwidth in Gaussian kernel. Since the hyper-parameters have a significant influence on the final performance of kernel SVM and are pretty hard to tune especially for large scale data, one may need to put lots of effort into finding good enough parameters, and improper settings of the hyper-parameters often make the classification performance even lower than that of linear SVM. Inspired by recent progresses on linear SVM for dealing with large scale data, we propose a well-designed classifier to efficiently handle large scale linearly inseparable data, i.e., Decision Tree SVM (DTSVM). DTSVM has much lower computational cost compared with kernel SVM, and it brings almost no hyper-parameters except a few thresholds which can be fixed in practice. Comprehensive experiments on large scale datasets demonstrate the superiority of the proposed method.