Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
复制标题
DOI：
--
复制
发表时间：
2017-07
期刊：
影响因子：
--
通讯作者：
John Bradshaw;A. G. Matthews;Zoubin Ghahramani
中科院分区：
文献类型：
--
作者：
John Bradshaw;A. G. Matthews;Zoubin Ghahramani
深度神经网络（DNN）具有出色的代表性能力，并且是许多任务上最先进的分类器。然而，他们往往不能很好地捕捉自己的不确定性，从而导致他们在现实世界中的鲁棒性较差，因为他们过度自信地推断并且没有注意到领域的转移。另一方面，具有 RBF 核的高斯过程 (GP) 具有更好的校准不确定性，并且不会过度自信地从训练集中的数据进行推断。然而，GP 的表征能力较差，并且在复杂领域上的表现不如 DNN。在本文中，我们展示了 GP 混合深度网络 GPDNN（DNN 之上的 GP 并经过端到端训练）继承了 GP 和 DNN 的优良特性，并且对于对抗性示例更加鲁棒。当外推到对抗性示例并在域转移设置中进行测试时，GPDNN 经常输出与本质上“不知道”相对应的高熵类别概率。因此，GPDNN 作为一种能够知道何时不知道的深度架构很有前景。
Deep neural networks (DNNs) have excellent representative power and are state of the art classifiers on many tasks. However, they often do not capture their own uncertainties well making them less robust in the real world as they overconfidently extrapolate and do not notice domain shift. Gaussian processes (GPs) with RBF kernels on the other hand have better calibrated uncertainties and do not overconfidently extrapolate far from data in their training set. However, GPs have poor representational power and do not perform as well as DNNs on complex domains. In this paper we show that GP hybrid deep networks, GPDNNs, (GPs on top of DNNs and trained end-to-end) inherit the nice properties of both GPs and DNNs and are much more robust to adversarial examples. When extrapolating to adversarial examples and testing in domain shift settings, GPDNNs frequently output high entropy class probabilities corresponding to essentially "don't know". GPDNNs are therefore promising as deep architectures that know when they don't know.