PremiUm-CNN: Propagating Uncertainty Towards Robust Convolutional Neural Networks
PremiUm-CNN: Propagating Uncertainty Towards Robust Convolutional Neural Networks
复制标题
优质卷积神经网络（PremiUm - CNN）：向稳健的卷积神经网络传播不确定性
DOI：
10.1109/tsp.2021.3096804
复制
发表时间：
2021
影响因子：
5.4
通讯作者：
Dimah Dera;N. Bouaynaya;G. Rasool;R. Shterenberg;H. Fathallah-Shaykh
中科院分区：
文献类型：
--
作者：
Dimah Dera;N. Bouaynaya;G. Rasool;R. Shterenberg;H. Fathallah-Shaykh
深度神经网络(DNN)在各种学习任务中的准确率已经超过了人类水平。然而，与人类对概率有着天然的认知直觉不同，DNN不能在输出决策中表达他们的不确定性。这限制了DNN在任务关键领域的部署，例如作战人员决策或医疗诊断。贝叶斯推理通过估计未知参数的后验分布，为模型不确定性的推理提供了一种原则性的方法。DNN中的挑战仍然是多层阶段的非线性，这使得高维分布的传播在数学上很难处理。本文提出了一种新的深度学习模型--Premium-CNN(在卷积神经网络中传播不确定性)，为不确定性或信念传播奠定了理论和算法基础。我们引入张量正态分布作为卷积核上的先验分布，并通过最大化证据下界(ELBO)来估计变分后验。我们首先推导出一阶均值-协方差传播框架。然后，我们开发了一个基于无迹变换(至少校正到二阶)的框架，该框架通过CNN的各层传播变分分布的sigma点。预测分布的传播协方差捕获了输出决策中的不确定性。在不同的基准数据集上进行的全面实验表明：1)对噪声和对手攻击具有卓越的鲁棒性，2)通过预测不确定性进行自我评估，这种不确定性随着噪声或攻击级别的增加而迅速增加，以及3)从环境噪声中检测目标攻击的能力。
Deep neural networks (DNNs) have surpassed human-level accuracy in various learning tasks. However, unlike humans who have a natural cognitive intuition for probabilities, DNNs cannot express their uncertainty in the output decisions. This limits the deployment of DNNs in mission-critical domains, such as warfighter decision-making or medical diagnosis. Bayesian inference provides a principled approach to reason about model's uncertainty by estimating the posterior distribution of the unknown parameters. The challenge in DNNs remains the multi-layer stages of non-linearities, which make the propagation of high-dimensional distributions mathematically intractable. This paper establishes the theoretical and algorithmic foundations of uncertainty or belief propagation by developing new deep learning models named PremiUm-CNNs (Propagating Uncertainty in Convolutional Neural Networks). We introduce a tensor normal distribution as a prior over convolutional kernels and estimate the variational posterior by maximizing the evidence lower bound (ELBO). We start by deriving the first-order mean-covariance propagation framework. Later, we develop a framework based on the unscented transformation (correct at least up to the second-order) that propagates sigma points of the variational distribution through layers of a CNN. The propagated covariance of the predictive distribution captures uncertainty in the output decision. Comprehensive experiments conducted on diverse benchmark datasets demonstrate: 1) superior robustness against noise and adversarial attacks, 2) self-assessment through predictive uncertainty that increases quickly with increasing levels of noise or attacks, and 3) an ability to detect a targeted attack from ambient noise.