Neural Bootstrapper
Neural Bootstrapper
复制标题
DOI：
--
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Minsuk Shin;Hyungjoon Cho;Sungbin Lim
中科院分区：
文献类型：
--
作者：
Minsuk Shin;Hyungjoon Cho;Sungbin Lim
Bootstrapping是不确定性量化的主要工具，其理论和计算特性已在统计和机器学习领域进行了研究。然而，由于其重复计算的性质，为神经网络实现自举过程所需的计算负担非常沉重，这一事实严重阻碍了这些过程在现代深度学习的不确定性估计中的实际应用。为了克服这些不便，我们提出了一个称为神经引导程序（NeuBoots）的程序。我们发现，NeuBoots稳定地生成有效的自举样本，与传统的自举相比，以最小的额外计算成本与所需的目标样本相一致。因此，NeuBoots使得构造神经网络输出的自举置信区间并量化其预测不确定性成为可能。我们还建议将NeuBoots用于深度卷积神经网络，以考虑其在图像分类任务中的实用性，包括校准、分布外样本检测和主动学习。实证结果表明，NeuBoots对于上述目的是非常有益的。
Bootstrapping has been a primary tool for uncertainty quantification, and their theoretical and computational properties have been investigated in the field of statistics and machine learning. However, due to its nature of repetitive computations, the computational burden required to implement bootstrap procedures for the neural network is painfully heavy, and this fact seriously hurdles the practical use of these procedures on the uncertainty estimation of modern deep learning. To overcome the inconvenience, we propose a procedure called \emph{Neural Bootstrapper} (NeuBoots). We reveal that the NeuBoots stably generate valid bootstrap samples that coincide with the desired target samples with minimal extra computational cost compared to traditional bootstrapping. Consequently, NeuBoots makes it feasible to construct bootstrap confidence intervals of outputs of neural networks and quantify their predictive uncertainty. We also suggest NeuBoots for deep convolutional neural networks to consider its utility in image classification tasks, including calibration, detection of out-of-distribution samples, and active learning. Empirical results demonstrate that NeuBoots is significantly beneficial for the above purposes.