Graph Reparameterizations for Enabling 1000+ Monte Carlo Iterations in Bayesian Deep Neural Networks.
Graph Reparameterizations for Enabling 1000+ Monte Carlo Iterations in Bayesian Deep Neural Networks.
复制标题
DOI：
--
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Singh V
中科院分区：
文献类型：
--
作者：
Nazarovs J;Mehta RR;Lokhande VS;Singh V
深度模型中的不确定性估计在许多现实世界的应用中是必不可少的，并且已经从过去几年的发展中受益。最近的证据表明，现有的解决方案依赖于简单的高斯公式可能是不够的。然而，转移到其他分布需要蒙特卡洛（MC）采样来估计KL散度等数量：随着输入数据和模型的维度增长，它可能是昂贵的，并且规模很小。这与计算图的结构直接相关，计算图可以作为所需MC样本数量的函数线性增长。在这里，我们构建了一个框架来描述这些计算图，并确定概率家庭的图形大小可以是独立的或只有弱依赖于MC样本的数量。这些族直接对应于大类的分布。从经验上讲，我们可以为计算机视觉中使用的大型架构运行更多的MC近似迭代，并在自信的准确性，训练稳定性，内存和训练时间方面获得性能提升。
Uncertainty estimation in deep models is essential in many real-world applications and has benefited from developments over the last several years. Recent evidence suggests that existing solutions dependent on simple Gaussian formulations may not be sufficient. However, moving to other distributions necessitates Monte Carlo (MC) sampling to estimate quantities such as the KL divergence: it could be expensive and scales poorly as the dimensions of both the input data and the model grow. This is directly related to the structure of the computation graph, which can grow linearly as a function of the number of MC samples needed. Here, we construct a framework to describe these computation graphs, and identify probability families where the graph size can be independent or only weakly dependent on the number of MC samples. These families correspond directly to large classes of distributions. Empirically, we can run a much larger number of iterations for MC approximations for larger architectures used in computer vision with gains in performance measured in confident accuracy, stability of training, memory and training time.