Second Order PAC-Bayesian Bounds for the Weighted Majority Vote
Second Order PAC-Bayesian Bounds for the Weighted Majority Vote
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
A. Masegosa;S. Lorenzen;C. Igel;Yevgeny Seldin
中科院分区：
文献类型：
--
作者：
A. Masegosa;S. Lorenzen;C. Igel;Yevgeny Seldin
我们提出了一种新的分析的预期风险加权多数表决多类分类。该分析考虑到合奏成员的预测的相关性，并提供了一个约束，这是服从有效的最小化，从而产生改善的加权多数票。我们还提供了一个专门版本的二进制分类，它允许利用额外的未标记数据进行更严格的风险估计。在实验中，我们应用的约束，以提高权重的树木在随机森林和显示，在常用的一阶界，最小化的新的界限通常不会导致退化的测试误差的合奏。
We present a novel analysis of the expected risk of weighted majority vote in multiclass classification. The analysis takes correlation of predictions by ensemble members into account and provides a bound that is amenable to efficient minimization, which yields improved weighting for the majority vote. We also provide a specialized version of our bound for binary classification, which allows to exploit additional unlabeled data for tighter risk estimation. In experiments, we apply the bound to improve weighting of trees in random forests and show that, in contrast to the commonly used first order bound, minimization of the new bound typically does not lead to degradation of the test error of the ensemble.