PAC-Bayes with Backprop
PAC-Bayes with Backprop
复制标题
具有反向传播的 PAC 贝叶斯
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Csaba Szepesvari
中科院分区：
文献类型：
--
作者：
Omar Rivasplata;Vikram Tankasali;Csaba Szepesvari
我们探索了一系列的方法“PAC-Bayes与Backprop”（PBB），通过最小化PAC-Bayes边界来训练概率神经网络。我们提出了两个训练目标，一个来自以前已知的PAC贝叶斯界，第二个来自一个新的PAC贝叶斯界。这两个培训目标进行评估MNIST和各种UCI数据集。我们的实验显示了两个惊人的观察：我们获得了有竞争力的测试集误差估计（在MNIST上约为1.4%），同时我们计算了比以前结果更严格的值（在MNIST上约为2.3%）的非空界。这些观察结果表明，由PBB训练的神经网络可能会导致自约束学习，其中可用数据可用于同时学习预测因子并验证其风险，而无需遵循数据拆分协议。
We explore the family of methods "PAC-Bayes with Backprop" (PBB) to train probabilistic neural networks by minimizing PAC-Bayes bounds. We present two training objectives, one derived from a previously known PAC-Bayes bound, and a second one derived from a novel PAC-Bayes bound. Both training objectives are evaluated on MNIST and on various UCI data sets. Our experiments show two striking observations: we obtain competitive test set error estimates (~1.4% on MNIST) and at the same time we compute non-vacuous bounds with much tighter values (~2.3% on MNIST) than previous results. These observations suggest that neural nets trained by PBB may lead to self-bounding learning, where the available data can be used to simultaneously learn a predictor and certify its risk, with no need to follow a data-splitting protocol.