Algorithms for Sparse Support Vector Machines.
Algorithms for Sparse Support Vector Machines.
复制标题
稀疏支持向量机算法。
DOI：
10.1080/10618600.2022.2146697
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Lange,Kenneth
中科院分区：
文献类型：
--
作者：
Landeros,Alfonso;Lange,Kenneth
分类中的许多问题涉及大量不相关的特征。变量选择揭示了关键特征，降低了特征空间的维数，提高了模型解释。在支持向量机文献中，变量选择是通过惩罚来实现的。这些凸松弛严重地使参数估计偏向于0，并倾向于承认太多不相关的特征。本文提出了一种替代方法，即用稀疏集约束代替惩罚。惩罚仍然存在，但目的不同。近距离原理采用损失函数L(β)，并将惩罚ρ2dist(β,Sk)2（捕获参数向量β的欧式距离的平方）添加到稀疏性集skk中，其中β的大多数分量是非零的。如果βρ代表目标fρ(β)=L(β)+ρ2dist(β,Sk)2的最小值，则βρ趋于L(β) overkasρ趋于约束的最小值。我们推导了两个密切相关的算法来实现这一策略。我们的仿真和实际示例生动地展示了算法如何在不损失分类能力的情况下获得更好的稀疏性。本文的补充材料可在网上获得。
Many problems in classification involve huge numbers of irrelevant features. Variable selection reveals the crucial features, reduces the dimensionality of feature space, and improves model interpretation. In the support vector machine literature, variable selection is achieved bypenalties. These convex relaxations seriously bias parameter estimates toward 0 and tend to admit too many irrelevant features. The current article presents an alternative that replaces penalties by sparse-set constraints. Penalties still appear, but serve a different purpose. The proximal distance principle takes a loss function L(β) and adds the penalty ρ2dist(β,Sk)2 capturing the squared Euclidean distance of the parameter vector β to the sparsity setSkwhere at mostkcomponents of β are nonzero. If βρ represents the minimum of the objective fρ(β)=L(β)+ρ2dist(β,Sk)2, then βρ tends to the constrained minimum of L(β) overSkasρtends to. We derive two closely related algorithms to carry out this strategy. Our simulated and real examples vividly demonstrate how the algorithms achieve better sparsity without loss of classification power. Supplementary materials for this article are available online.