Hierarchical architectures in reservoir computing systems
Hierarchical architectures in reservoir computing systems
复制标题
DOI：
10.1088/2634-4386/ac1b75
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
John Moon;Yuting Wu;Wei D. Lu
中科院分区：
文献类型：
--
作者：
John Moon;Yuting Wu;Wei D. Lu
通过将递归神经网络分解为具有递归连接的固定网络和可训练的线性网络，水库计算(RC)以低训练成本提供了高效的时态数据处理。固定网络的质量是决定RC系统性能的最重要因素。在本文中，我们研究了分级水库结构对水库特性和RC系统性能的影响。类似于深度神经网络，串联叠加子油藏是增强数据向高维空间转换的非线性，扩大油藏捕获的时间信息的多样性的有效方法。与简单地增加储集层的大小或子储集层的数量相比，这些深层储集层系统提供了更好的性能。低频分量主要由深层储集层构造后期的子储集层捕获，类似于观测结果表明，在深层神经网络的后期，可以分层提取更抽象的信息。当水库总库容固定时，由于个别小规模的子水库的退化能力，需要仔细考虑子水库的数量和每个子水库的大小之间的权衡。深层水库结构性能的提高缓解了在硬件系统上实施RC系统的难度。
Reservoir computing (RC) offers efficient temporal data processing with a low training cost by separating recurrent neural networks into a fixed network with recurrent connections and a trainable linear network. The quality of the fixed network, called reservoir, is the most important factor that determines the performance of the RC system. In this paper, we investigate the influence of the hierarchical reservoir structure on the properties of the reservoir and the performance of the RC system. Analogous to deep neural networks, stacking sub-reservoirs in series is an efficient way to enhance the nonlinearity of data transformation to high-dimensional space and expand the diversity of temporal information captured by the reservoir. These deep reservoir systems offer better performance when compared to simply increasing the size of the reservoir or the number of sub-reservoirs. Low frequency components are mainly captured by the sub-reservoirs in later stage of the deep reservoir structure, similar to observations that more abstract information can be extracted by layers in the late stage of deep neural networks. When the total size of the reservoir is fixed, tradeoff between the number of sub-reservoirs and the size of each sub-reservoir needs to be carefully considered, due to the degraded ability of individual sub-reservoirs at small sizes. Improved performance of the deep reservoir structure alleviates the difficulty of implementing the RC system on hardware systems.