Boosting in the Presence of Outliers: Adaptive Classification With Nonconvex Loss Functions
Boosting in the Presence of Outliers: Adaptive Classification With Nonconvex Loss Functions
复制标题
DOI：
10.1080/01621459.2016.1273116
复制
发表时间：
2015-10
影响因子：
3.7
通讯作者：
Alexander Hanbo Li;Jelena Bradic
中科院分区：
文献类型：
--
作者：
Alexander Hanbo Li;Jelena Bradic
摘要本文研究了非凸损失函数在二分类问题中的作用和效率。特别是，我们研究了如何设计自适应和有效的Boosting算法，该算法对数据中存在的离群值或观察到的数据标签中存在的错误具有健壮性。我们证明了非凸损失对预测精度起着重要的作用，因为它具有递减的梯度性质--损失能够有效地适应外围数据。我们提出了一种称为ArchBoost的新的Boost框架，它直接利用递减的梯度性质，从而产生可证明是健壮的Boost算法。在ArchBoost框架的基础上，提出了一类非凸损失，由此提出了一种新的稳健增强算法，称为自适应稳健增强算法(ARB)。此外，我们还发展了一种新的故障点分析和一种新的影响函数分析，证明了该方法在稳健性方面的优势。此外，仅基于局部曲率，我们建立了所提出的具有高度非凸损失的ArchBoost算法的统计和优化性质。大量的数值和真实数据例子说明了理论性质，并显示了在数据受到对手干扰或其他情况下优于现有的增强方法。这篇文章的补充材料可以在网上找到。
ABSTRACT This article examines the role and the efficiency of nonconvex loss functions for binary classification problems. In particular, we investigate how to design adaptive and effective boosting algorithms that are robust to the presence of outliers in the data or to the presence of errors in the observed data labels. We demonstrate that nonconvex losses play an important role for prediction accuracy because of the diminishing gradient properties—the ability of the losses to efficiently adapt to the outlying data. We propose a new boosting framework called ArchBoost that uses diminishing gradient property directly and leads to boosting algorithms that are provably robust. Along with the ArchBoost framework, a family of nonconvex losses is proposed, which leads to the new robust boosting algorithms, named adaptive robust boosting (ARB). Furthermore, we develop a new breakdown point analysis and a new influence function analysis that demonstrate gains in robustness. Moreover, based only on local curvatures, we establish statistical and optimization properties of the proposed ArchBoost algorithms with highly nonconvex losses. Extensive numerical and real data examples illustrate theoretical properties and reveal advantages over the existing boosting methods when data are perturbed by an adversary or otherwise. Supplementary materials for this article are available online.