Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures
Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures
复制标题
DOI：
--
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Yuan Cao;Quanquan Gu;M. Belkin
中科院分区：
文献类型：
--
作者：
Yuan Cao;Quanquan Gu;M. Belkin
现代机器学习系统（例如深度神经网络）通常高度过度参数化，以便它们可以准确地拟合噪声训练数据，但在实践中仍然可以实现较小的测试误差。在本文中，我们研究了线性分类问题的最大间隔分类器的这种“良性过拟合”现象。具体来说，我们考虑从亚高斯混合生成的数据，并为过度参数化设置中的最大边缘线性分类器提供严格的风险界限。我们的结果精确地描述了线性分类问题中发生良性过拟合的条件，并改进了之前的工作。它们还对过度参数化的逻辑回归有直接影响。
Modern machine learning systems such as deep neural networks are often highly over-parameterized so that they can fit the noisy training data exactly, yet they can still achieve small test errors in practice. In this paper, we study this"benign overfitting"phenomenon of the maximum margin classifier for linear classification problems. Specifically, we consider data generated from sub-Gaussian mixtures, and provide a tight risk bound for the maximum margin linear classifier in the over-parameterized setting. Our results precisely characterize the condition under which benign overfitting can occur in linear classification problems, and improve on previous work. They also have direct implications for over-parameterized logistic regression.