PAC-Bayesian Bounds on Rate-Efficient Classifiers
PAC-Bayesian Bounds on Rate-Efficient Classifiers
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Alhabib Abbas;Y. Andreopoulos
中科院分区：
文献类型：
--
作者：
Alhabib Abbas;Y. Andreopoulos
我们得出了在压缩输入上运行的多数投票分类器的噪声不变性的分析界限。具体来说，从最近多数投票分类器真实风险的界限开始，我们扩展了 PAC-贝叶斯理论的适用性，以量化多数投票对压缩产生的输入噪声的弹性。导出的界限在二元分类设置中是直观的，可以将它们作为选民差异和选民对一致性的表达式进行测量。通过将输入失真的测量与噪声不变性的分析保证相结合，我们规定了速率高效的机器来压缩输入而不影响后续分类。我们的验证显示了边界噪声不变性如何为任何多数投票分类器的压缩阶段提供信息，以便了解不良输入重建的最坏情况影响，并且可以将输入压缩到推理之前所需的最小信息量。
We derive analytic bounds on the noise invariance of majority vote classifiers operating on compressed inputs. Specifically, starting from recent bounds on the true risk of majority vote classifiers, we extend the applicability of PAC-Bayesian theory to quantify the resilience of majority votes to input noise stemming from compression. The derived bounds are intuitive in binary classification settings, where they can be measured as expressions of voter differentials and voter pair agreement. By combining measures of input distortion with analytic guarantees on noise invariance, we prescribe rate-efficient machines to compress inputs without affecting subsequent classification. Our validation shows how bounding noise invariance can inform the compression stage for any majority vote classifier such that worst-case implications of bad input reconstructions are known, and inputs can be compressed to the minimum amount of information needed prior to inference.