On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification
On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification
复制标题
DOI：
10.48550/arxiv.2203.16481
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Sanyam Kapoor;Wesley J. Maddox;Pavel Izmailov;A. Wilson
中科院分区：
文献类型：
--
作者：
Sanyam Kapoor;Wesley J. Maddox;Pavel Izmailov;A. Wilson
随机不确定性捕获数据的固有随机性，例如测量噪声。在贝叶斯回归中，我们经常使用高斯观测模型，其中我们用噪声方差参数控制任意不确定性的水平。相比之下，对于贝叶斯分类，我们使用分类分布，没有机制来表示我们对任意不确定性的信念。我们的工作表明，明确考虑任意的不确定性显着提高贝叶斯神经网络的性能。我们注意到，许多标准基准，如CIFAR，基本上没有任意的不确定性。此外，我们发现近似推理中的数据增强具有软化可能性的效果，导致信心不足，并深刻地歪曲了我们对任意不确定性的诚实信念。因此，我们发现，一个冷的后验，由一个权力大于一，往往更诚实地反映了我们的信念比没有回火任意不确定性-提供了一个明确的数据增强和冷后验之间的联系。我们表明，我们可以匹配或超过后回火的性能，通过使用狄利克雷观测模型，在那里我们明确地控制任意的不确定性的水平，而不需要任何回火。
Aleatoric uncertainty captures the inherent randomness of the data, such as measurement noise. In Bayesian regression, we often use a Gaussian observation model, where we control the level of aleatoric uncertainty with a noise variance parameter. By contrast, for Bayesian classification we use a categorical distribution with no mechanism to represent our beliefs about aleatoric uncertainty. Our work shows that explicitly accounting for aleatoric uncertainty significantly improves the performance of Bayesian neural networks. We note that many standard benchmarks, such as CIFAR, have essentially no aleatoric uncertainty. Moreover, we show data augmentation in approximate inference has the effect of softening the likelihood, leading to underconfidence and profoundly misrepresenting our honest beliefs about aleatoric uncertainty. Accordingly, we find that a cold posterior, tempered by a power greater than one, often more honestly reflects our beliefs about aleatoric uncertainty than no tempering -- providing an explicit link between data augmentation and cold posteriors. We show that we can match or exceed the performance of posterior tempering by using a Dirichlet observation model, where we explicitly control the level of aleatoric uncertainty, without any need for tempering.