Ensemble echo network with deep architecture for time-series modeling
Ensemble echo network with deep architecture for time-series modeling
复制标题
具有深度架构的集成回波网络，用于时间序列建模
DOI：
10.1007/s00521-020-05286-8
复制
发表时间：
2020
影响因子：
6
通讯作者：
Chang Sheng
中科院分区：
文献类型：
--
作者：
Hu Ruihan;Tang Zhi-Ri;Song Xiaoying;Luo Jun;Wu Edmond Q.;Chang Sheng
回声状态网络是一种被广泛应用于时间序列数据建模的递归神经网络。回声状态网络中的储集器功能期望从时间序列数据集中提取特征上下文。然而，由于回声状态网络的结构是固定的，且超参数难以自动确定，因此其在实际应用中的推广受到了限制。提出了一种具有深度和灵活性结构的集成贝叶斯深度回声网络模型。这种具有深部结构的网络通过多个储集层逐渐提取比具有浅储集层的网络更多的动态回波状态。为了增强网络配置的灵活性，研究了超参数的贝叶斯优化过程，确定了合适的超参数来激活网络。此外，在处理更复杂的时间序列数据集时，Ebden的集成机制可以在不牺牲算法性能的情况下度量时间序列通道的冗余度。本文通过在多变量时间序列知识库和实际任务(如混沌序列表示和Dansgaard-Oeschger估计任务)上的实验，验证了Ebden的深度、优化和集成结构。结果表明，与最先进的模型相比，Ebden获得了较高的拟合优度和分类性能。
Echo state network belongs to a kind of recurrent neural networks that have been extensively employed to model time-series datasets. The function of reservoir in echo state network is expected to extract the feature context from time-series datasets. However, generalization of echo state networks is limited in real-world application because the architectures of the network are fixed and the hyper-parameters are hard to be automatically determined. In the present study, the ensemble Bayesian deep echo network (EBDEN) model with deep and flexible architecture is proposed. Such networks with deep architecture progressively extract more dynamic echo states through multiple reservoirs than those with the shallow reservoir. To enhance the flexibility of the configuration for the network, this study investigates the Bayesian optimization procedure of hyper-parameters and ensures the suitable hyper-parameters to activate the network. In addition, when dealing with more complex time-series datasets, ensemble mechanism of EBDEN can measure the redundancy for the channels of the time series without sacrificing the algorithm’s performance. In this paper, the deep, optimization and ensemble architectures of EBDEN are verified by experiments benchmarked on multivariate time-series repositories and realistic tasks such as chaotic series representation and Dansgaard–Oeschger estimation tasks. According to the results, EBDEN achieves high level of the goodness-of-fit and classification performance in comparison with state-of-the-art models.