Rényi Divergence Based Bounds on Generalization Error
Rényi Divergence Based Bounds on Generalization Error
复制标题
基于 Rényi 散度的泛化误差界限
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
V. Prabhakaran
中科院分区：
文献类型：
--
作者：
Eeshan Modak;Himanshu Asnani;V. Prabhakaran
泛化误差捕获学习算法的输出与训练数据过度拟合的程度。我们获得了一系列界限，它概括了 Xu 和 Raginsky (2017) 以及 Bu、Zou 和 Veeravalli (2019) 在某些假设下开发的界限。我们的界限基于 Kullback-Leibler 散度的 Donsker-Varadhan 表示的 Rényi 模拟。我们还获得了泛化误差概率的界限，恢复了 Esposito、Gastpar 和 Issa (2020) 的界限。我们还给出了 0-1 损失函数的预期真实损失的乘法下界。
Generalization error captures the degree to which the output of a learning algorithm overfits the training data. We obtain a family of bounds which generalize the bounds developed by Xu & Raginsky (2017) and Bu, Zou and Veeravalli (2019), under certain assumptions. Our bounds are based on the Rényi analogue of the Donsker-Varadhan representation of Kullback-Leibler divergence. We also obtain bounds on the probability of generalization error which recover the bounds of Esposito, Gastpar and Issa (2020). We also give a multiplicative lower bound on the expected true loss for a 0-1 loss function.