Robust Learning via Ensemble Density Propagation in Deep Neural Networks
Robust Learning via Ensemble Density Propagation in Deep Neural Networks
复制标题
DOI：
10.1109/mlsp49062.2020.9231635
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Giuseppina Carannante;Dimah Dera;G. Rasool;N. Bouaynaya;L. Mihaylova
中科院分区：
文献类型：
--
作者：
Giuseppina Carannante;Dimah Dera;G. Rasool;N. Bouaynaya;L. Mihaylova
在不确定、噪声或对抗性环境中学习对于深度神经网络（DNN）来说是一项具有挑战性的任务。我们提出了一种新的理论基础和有效的方法，建立在贝叶斯估计和变分推理的鲁棒学习。我们制定了通过DNN层的密度传播问题，并使用Ensemble Density Propagation（EnDP）方案解决它。EnDP方法允许我们在贝叶斯DNN的各层中传播变分概率分布的矩，从而能够估计模型输出的预测分布的均值和协方差。我们使用MNIST和CIFAR-10数据集进行的实验显示，训练模型对随机噪声和对抗性攻击的鲁棒性显着提高。
Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.