Multiple Descent: Design Your Own Generalization Curve
Multiple Descent: Design Your Own Generalization Curve
复制标题
DOI：
--
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Lin Chen;Yifei Min;M. Belkin;Amin Karbasi
中科院分区：
文献类型：
--
作者：
Lin Chen;Yifei Min;M. Belkin;Amin Karbasi
本文探讨了线性回归在欠参数化和过参数化模型族中的推广损失。我们发现，泛化曲线可以有任意数量的峰值，而且，这些峰值的位置可以明确控制。我们的结果突出了一个事实，即经典的U形推广曲线和最近观察到的双下降曲线不是模型族的内在属性。相反，它们的出现是由于数据属性和学习算法的归纳偏差之间的相互作用。
This paper explores the generalization loss of linear regression in variably parameterized families of models, both under-parameterized and over-parameterized. We show that the generalization curve can have an arbitrary number of peaks, and moreover, locations of those peaks can be explicitly controlled. Our results highlight the fact that both classical U-shaped generalization curve and the recently observed double descent curve are not intrinsic properties of the model family. Instead, their emergence is due to the interaction between the properties of the data and the inductive biases of learning algorithms.