A Precise Performance Analysis of Learning with Random Features
A Precise Performance Analysis of Learning with Random Features
复制标题
随机特征学习的精确性能分析
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Yue M. Lu
中科院分区：
文献类型：
--
作者：
Oussama Dhifallah;Yue M. Lu
我们研究了使用随机特征模型学习未知函数的问题。我们的主要贡献是对这类具有高斯数据的学习问题进行了精确的渐近分析。在特征矩阵的温和正则性条件下，我们给出了渐近训练误差和泛化误差的精确刻画，在欠参数和过参数情况下都是有效的。本文的分析适用于一般的特征矩阵族、激活函数族和凸损失函数族。数值结果验证了我们的理论预测，表明我们的渐近结果与所考虑的学习问题的实际性能非常吻合，即使在中等维度上也是如此。此外，它们还揭示了正则化、损失函数和激活函数在缓解学习中的“双下降现象”中所起的重要作用。
We study the problem of learning an unknown function using random feature models. Our main contribution is an exact asymptotic analysis of such learning problems with Gaussian data. Under mild regularity conditions for the feature matrix, we provide an exact characterization of the asymptotic training and generalization errors, valid in both the under-parameterized and over-parameterized regimes. The analysis presented in this paper holds for general families of feature matrices, activation functions, and convex loss functions. Numerical results validate our theoretical predictions, showing that our asymptotic findings are in excellent agreement with the actual performance of the considered learning problem, even in moderate dimensions. Moreover, they reveal an important role played by the regularization, the loss function and the activation function in the mitigation of the "double descent phenomenon" in learning.
影响因子：
3
作者：
Song Mei;A. Montanari
通讯作者：
Song Mei;A. Montanari
DOI：
10.1073/pnas.1903070116
发表时间：
2019-08-06
影响因子：
11.1
作者：
Belkin, Mikhail;Hsu, Daniel;Mandal, Soumik
通讯作者：
Mandal, Soumik