Smoothly Giving up: Robustness for Simple Models
Smoothly Giving up: Robustness for Simple Models
复制标题
DOI：
10.48550/arxiv.2302.09114
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Tyler Sypherd;Nathan Stromberg;R. Nock;Visar Berisha;L. Sankar
中科院分区：
文献类型：
--
作者：
Tyler Sypherd;Nathan Stromberg;R. Nock;Visar Berisha;L. Sankar
人们越来越需要可解释的、降低能耗和计算成本的模型（例如，在医疗保健分析和联邦学习中）。训练这种模型的算法示例包括逻辑回归和增强。然而，这些算法面临的一个挑战是它们明显受到标签噪声的影响；这归因于经常使用的凸损失函数和更简单的假设类之间的联合相互作用，导致过分强调离群值。在这项工作中，我们使用基于边缘的$\alpha$-loss，它在正则凸和准凸损失之间连续调谐，以鲁棒性训练简单模型。我们证明了$\alpha$超参数平滑地引入了非凸性，并提供了“放弃”噪声训练示例的好处。我们还提供了用于增强的Long-Servedio数据集和用于逻辑回归的COVID-19调查数据集的结果，突出了我们的方法在多个相关领域的有效性。
There is a growing need for models that are interpretable and have reduced energy and computational cost (e.g., in health care analytics and federated learning). Examples of algorithms to train such models include logistic regression and boosting. However, one challenge facing these algorithms is that they provably suffer from label noise; this has been attributed to the joint interaction between oft-used convex loss functions and simpler hypothesis classes, resulting in too much emphasis being placed on outliers. In this work, we use the margin-based $\alpha$-loss, which continuously tunes between canonical convex and quasi-convex losses, to robustly train simple models. We show that the $\alpha$ hyperparameter smoothly introduces non-convexity and offers the benefit of "giving up" on noisy training examples. We also provide results on the Long-Servedio dataset for boosting and a COVID-19 survey dataset for logistic regression, highlighting the efficacy of our approach across multiple relevant domains.