Attention-Based SeriesNet: An Attention-Based Hybrid Neural Network Model for Conditional Time Series Forecasting
Attention-Based SeriesNet: An Attention-Based Hybrid Neural Network Model for Conditional Time Series Forecasting
复制标题
DOI：
10.3390/info11060305
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Yepeng Cheng;Zuren Liu;Y. Morimoto
中科院分区：
文献类型：
--
作者：
Yepeng Cheng;Zuren Liu;Y. Morimoto
传统的时间序列预测技术不能提取足够好的序列数据特征，预测精度有限。深度学习结构SeriesNet是一种先进的方法，它采用混合神经网络，包括扩张因果卷积神经网络（DC-CNN）和长短期记忆递归神经网络（LSTM-RNN），以更高的精度从多条件时间序列中学习多范围和多水平特征。然而，他们没有考虑注意机制来学习时间特征。此外，CNN和RNN的条件化方法并不具体，每层的参数数量巨大。本文提出了两种神经网络的条件化方法，分别用门控递归单元网络（GRU）和扩张的深度可分离时间卷积网络（DDSTCN）代替LSTM和DC-CNN进行参数约简。此外，本文提出了轻量级的基于RNN的隐藏状态注意力模块（HSAM）结合提出的基于CNN的卷积块注意力模块（CBAM）的时间序列预测。实验结果表明，该模型在预测精度和计算效率方面均优于其他模型，具有上级的特点。
Traditional time series forecasting techniques can not extract good enough sequence data features, and their accuracies are limited. The deep learning structure SeriesNet is an advanced method, which adopts hybrid neural networks, including dilated causal convolutional neural network (DC-CNN) and Long-short term memory recurrent neural network (LSTM-RNN), to learn multi-range and multi-level features from multi-conditional time series with higher accuracy. However, they didn’t consider the attention mechanisms to learn temporal features. Besides, the conditioning method for CNN and RNN is not specific, and the number of parameters in each layer is tremendous. This paper proposes the conditioning method for two types of neural networks, and respectively uses the gated recurrent unit network (GRU) and the dilated depthwise separable temporal convolutional networks (DDSTCNs) instead of LSTM and DC-CNN for reducing the parameters. Furthermore, this paper presents the lightweight RNN-based hidden state attention module (HSAM) combined with the proposed CNN-based convolutional block attention module (CBAM) for time series forecasting. Experimental results show our model is superior to other models from the viewpoint of forecasting accuracy and computation efficiency.