Frank-Wolfe algorithm for learning SVM-type multi-category classifiers
Frank-Wolfe algorithm for learning SVM-type multi-category classifiers
复制标题
DOI：
10.1587/transinf.2021edp7025
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Kenya Tajima;Yoshihiro Hirohashi;E. R. R. Zara-E.-R.-R.-Zara-70828201;Tsuyoshi Kato
中科院分区：
文献类型：
--
作者：
Kenya Tajima;Yoshihiro Hirohashi;E. R. R. Zara-E.-R.-R.-Zara-70828201;Tsuyoshi Kato
多类别支持向量机（MC-SVM）是最流行的机器学习算法之一。尽管针对不同的学习机开发了不同的优化算法，但 MC-SVM 有很多变体。在这项研究中，我们开发了一种新的优化算法，可以应用于许多 MC-SVM 变体。该算法基于 Frank-Wolfe 框架，在每次迭代中需要两个子问题：方向查找和线搜索。这项研究的贡献在于发现，如果将 Frank-Wolfe 框架应用于对偶问题，两个子问题都有封闭形式的解。此外，即使对于损失函数的莫罗包络，也存在测向和线搜索的封闭形式解。我们使用几个大型数据集来证明所提出的优化算法快速收敛，从而提高了模式识别性能。
Multi-category support vector machine (MC-SVM) is one of the most popular machine learning algorithms. There are lots of variants of MC-SVM, although different optimization algorithms were developed for different learning machines. In this study, we developed a new optimization algorithm that can be applied to many of MC-SVM variants. The algorithm is based on the Frank-Wolfe framework that requires two subproblems, direction finding and line search, in each iteration. The contribution of this study is the discovery that both subproblems have a closed form solution if the Frank-Wolfe framework is applied to the dual problem. Additionally, the closed form solutions on both for the direction finding and for the line search exist even for the Moreau envelopes of the loss functions. We use several large datasets to demonstrate that the proposed optimization algorithm converges rapidly and thereby improves the pattern recognition performance.