SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates
SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Lingkai Kong;Jimeng Sun;Chao Zhang
中科院分区：
文献类型：
--
作者：
Lingkai Kong;Jimeng Sun;Chao Zhang
不确定性量化是深度学习的一个基本但尚未解决的问题。贝叶斯框架提供了一种原则性的不确定性估计方法，但通常无法扩展到具有大量参数的现代深度神经网络 (DNN)。非贝叶斯方法实现起来很简单，但经常会混淆不同的不确定性来源，并且需要大量的计算资源。我们提出了一种从动力系统角度量化 DNN 不确定性的新方法。我们方法的核心是将 DNN 变换视为随机动力系统的状态演化，并引入布朗运动项来捕获认知不确定性。基于这个观点，我们提出了一种神经随机微分方程模型（SDE-Net），它由以下部分组成：（1）控制系统拟合预测函数的漂移网络； (2) 捕捉认知不确定性的扩散网。我们从理论上分析了SDE-Net解的存在性和唯一性。我们的实验表明，SDE-Net 模型可以在一系列不确定性起着基本作用的任务中优于现有的不确定性估计方法。
Uncertainty quantification is a fundamental yet unsolved problem for deep learning. The Bayesian framework provides a principled way of uncertainty estimation but is often not scalable to modern deep neural nets (DNNs) that have a large number of parameters. Non-Bayesian methods are simple to implement but often conflate different sources of uncertainties and require huge computing resources. We propose a new method for quantifying uncertainties of DNNs from a dynamical system perspective. The core of our method is to view DNN transformations as state evolution of a stochastic dynamical system and introduce a Brownian motion term for capturing epistemic uncertainty. Based on this perspective, we propose a neural stochastic differential equation model (SDE-Net) which consists of (1) a drift net that controls the system to fit the predictive function; and (2) a diffusion net that captures epistemic uncertainty. We theoretically analyze the existence and uniqueness of the solution to SDE-Net. Our experiments demonstrate that the SDE-Net model can outperform existing uncertainty estimation methods across a series of tasks where uncertainty plays a fundamental role.