Memory in linear recurrent neural networks in continuous time
Memory in linear recurrent neural networks in continuous time
复制标题
DOI：
10.1016/j.neunet.2009.08.008
复制
发表时间：
2010-04-01
期刊：
影响因子：
7.8
通讯作者：
Schrauwen, Benjamin
中科院分区：
文献类型：
--
作者：
Hermans, Michiel;Schrauwen, Benjamin
水库计算是一种新的技术，它采用递归神经网络，同时绕过困难的训练算法。油藏计算的一个非常新的趋势是使用真实的物理动态系统作为实现平台，而不是传统的数字仿真。物理系统在连续时间中运行，这与水库计算的经典离散时间定义有根本区别。本文的具体目标是研究这种系统的记忆特性，我们将限制自己的线性动力学。我们开发了一个分析模型，它允许计算连续时间线性动力系统的记忆函数，它可以被认为是线性漏积分神经元网络。然后，我们使用这个模型来研究不同类型的储层的记忆特性。我们从具有移位特征值谱的随机连接矩阵开始，其表现非常差。接下来，我们将两个特定的油藏类型，这是已知的，以提供良好的性能在离散时间，连续时间域。基于离散时间内单位圆盘上连接矩阵特征值均匀分布的水库比随机连接矩阵的水库具有更好的记忆性能，其中基于离散时间内正交连接矩阵的水库对噪声非常鲁棒，并且它们的记忆性能可以调节。在这项工作中发现的总体结果产生了重要的见解，如何设计网络的连续时间。(C)2009爱思唯尔有限公司版权所有。
Reservoir Computing is a novel technique which employs recurrent neural networks while circumventing difficult training algorithms. A very recent trend in Reservoir Computing is the use of real physical dynamical systems as implementation platforms, rather than the customary digital emulations. Physical systems operate in continuous time, creating a fundamental difference with the classic discrete time definitions of Reservoir Computing. The specific goal of this paper is to study the memory properties of such systems, where we will limit ourselves to linear dynamics. We develop an analytical model which allows the calculation of the memory function for continuous time linear dynamical systems, which can be considered as networks of linear leaky integrator neurons. We then use this model to research memory properties for different types of reservoir. We start with random connection matrices with a shifted eigenvalue spectrum, which perform very poorly. Next, we transform two specific reservoir types, which are known to give good performance in discrete time, to the continuous time domain. Reservoirs based on uniform spreading of connection matrix eigenvalues on the unit disk in discrete time give much better memory properties than reservoirs with random connection matrices, where reservoirs based on orthogonal connection matrices in discrete time are very robust against noise and their memory properties can be tuned. The overall results found in this work yield important insights into how to design networks for continuous time. (C) 2009 Elsevier Ltd. All rights reserved.