Triple-Stage Attention-Based Multiple Parallel Connection Hybrid Neural Network Model for Conditional Time Series Forecasting
Triple-Stage Attention-Based Multiple Parallel Connection Hybrid Neural Network Model for Conditional Time Series Forecasting
复制标题
DOI：
10.1109/access.2021.3059861
复制
发表时间：
2021-01-01
期刊：
影响因子：
3.9
通讯作者：
Morimoto, Yasuhiko
中科院分区：
文献类型：
--
作者：
Cheng, Yepeng;Morimoto, Yasuhiko
基于注意力的SeriesNet（A-SeriesNet）将基于增强注意力残差学习模块的卷积神经网络（augmented ARLM-CNN）子网络与基于隐藏状态注意力模块的递归神经网络（HSAM-RNN）子网络相结合，用于高精度的条件时间序列预测。增强的 ARLM-CNN 子网络在提取多条件序列的潜在特征方面存在缺陷。当多条件序列的特征维度变高时，预测精度会降低。同样的问题也出现在A-SeriesNet的HSAM-RNN子网络中。双阶段注意力循环神经网络（DA-RNN）证明基于注意力的编码器-解码器框架是处理上述问题的有效模型。本文将 DA-RNN 应用于 A-SeriesNet 的 HSAM-RNN 子网络，并提出了三阶段基于注意力的循环神经网络（TA-RNN）子网络。此外，本文考虑了一种基于 CNN 的编码器-解码器结构，称为基于双重注意力残差学习模块的卷积神经网络 (DARLM-CNN) 子网络，以改进 A-SeriesNet 的增强 ARLM-CNN 子网络。最后，本文提出了基于三阶段注意力的SeriesNet（TA-SeriesNet），它使用一种新的级联方法代替A-SeriesNet的逐元素乘法来并行连接所提出的子网络，并减少预测结果对某个子网络的依赖。实验结果表明，我们的 TA-SeriesNet 在高特征维度时间序列数据集的预测准确性评估指标方面优于其他深度学习模型。
The attention-based SeriesNet (A-SeriesNet) combined augmented attention residual learning module-based convolutional neural network (augmented ARLM-CNN) subnetwork with hidden state attention module-based recurrent neural network (HSAM-RNN) subnetwork for conditional time series prediction with high accuracy. The augmented ARLM-CNN subnetwork has defects in extracting latent features of the multi-condition series. The forecasting accuracy will decrease when the feature dimension of the multi-condition series becomes high. The same problem also occurs in the HSAM-RNN subnetwork of A-SeriesNet. The dual-stage attention recurrent neural network (DA-RNN) proved that the attention-based encoder-decoder framework is an effective model for dealing with the above problem. This paper applies the DA-RNN to the HSAM-RNN subnetwork of A-SeriesNet and presents the triple-stage attention-based recurrent neural network (TA-RNN) subnetworks. Furthermore, this paper considers a CNN-based encoder-decoder structure named dual attention residual learning module-based convolutional neural network (DARLM-CNN) subnetwork to improve the augmented ARLM-CNN subnetwork of A-SeriesNet. Finally, this paper presents the triple-stage attention-based SeriesNet (TA-SeriesNet), which uses a new concatenation method instead of the element-wise multiplication of A-SeriesNet to parallel connect the proposed subnetworks and reduce the dependence of forecasting results on a certain subnetwork. The experimental results show our TA-SeriesNet is superior to other deep learning models in forecasting accuracy evaluation metrics for high feature dimensional time series datasets.