TRustworthy Uncertainty Propagation for Sequential Time-Series Analysis in RNNs
TRustworthy Uncertainty Propagation for Sequential Time-Series Analysis in RNNs
复制标题
DOI：
10.1109/tkde.2023.3288628
复制
发表时间：
2024-02
影响因子：
8.9
通讯作者：
Dimah Dera;Sabeen Ahmed;N. Bouaynaya;G. Rasool
中科院分区：
文献类型：
--
作者：
Dimah Dera;Sabeen Ahmed;N. Bouaynaya;G. Rasool
通过物联网和数字医疗产生的海量时间序列需要新颖的数据建模和预测。递归神经网络(RNN)被广泛应用于时间序列数据的分析。然而，这些模型无法评估预测的不确定性，这在异质和噪声环境中尤为关键。贝叶斯推理允许通过估计参数的后验分布来对预测不确定性进行推理。挑战仍然是通过RNN的连续的、非线性的层传播高维分布，导致模式崩溃导致错误的不确定性估计并加剧梯度爆炸问题。通过引入网络参数上的高斯先验，并利用证据下界估计高斯变差分布的前两阶矩，提出了一种用于RNN中序贯时间序列分析(TRUST)的可信不确定性传播方法。我们使用一阶Taylor近似将变分矩传播到RNN的顺序、非线性层中。预测分布的传播协方差捕获了输出决策中的不确定性。使用ECG5000和PeMS-SF分类以及天气和功耗预测任务的广泛实验表明，1)信任-RNN对噪声和对手攻击具有显著的稳健性，2)通过随着噪声的增加而显著增加的不确定性进行自我评估。
The massive time-series production through the Internet of Things and digital healthcare requires novel data modeling and prediction. Recurrent neural networks (RNNs) are extensively used for analyzing time-series data. However, these models are unable to assess prediction uncertainty, which is particularly critical in heterogeneous and noisy environments. Bayesian inference allows reasoning about predictive uncertainty by estimating the posterior distribution of the parameters. The challenge remains in propagating the high-dimensional distribution through the sequential, non-linear layers of RNNs, resulting in mode collapse leading to erroneous uncertainty estimation and exacerbating the gradient explosion problem. This paper proposes a TRustworthy Uncertainty propagation for Sequential Time-series analysis (TRUST) in RNNs by introducing a Gaussian prior over network parameters and estimating the first two moments of the Gaussian variational distribution using the evidence lower bound. We propagate the variational moments through the sequential, non-linear layers of RNNs using the first-order Taylor approximation. The propagated covariance of the predictive distribution captures uncertainty in the output decision. The extensive experiments using ECG5000 and PeMS-SF classification and weather and power consumption prediction tasks demonstrate 1) significant robustness of TRUST-RNNs against noise and adversarial attacks and 2) self-assessment through the uncertainty that increases significantly with increasing noise.