Learning precise timing with LSTM recurrent networks
Learning precise timing with LSTM recurrent networks
复制标题
DOI：
10.1162/153244303768966139
复制
发表时间：
2003-01-01
影响因子：
6
通讯作者：
Schmidhuber, J
中科院分区：
文献类型：
--
作者：
Gers, FA;Schraudolph, NN;Schmidhuber, J
事件之间的时间距离传达了许多顺序任务（例如运动控制和节奏检测）所必需的信息。尽管隐藏的马尔可夫模型倾向于忽略这些信息，但经常性的神经网络（RNN）可以原则上学会利用它。我们专注于长期记忆（LSTM），因为已证明它在涉及长时间滞后的任务上表现优于其他RNN。我们发现，从其内部细胞到其乘法大门的“窥视孔连接”增强的LSTM可以学习间隔50或49个时间步长的序列之间的良好区别，而无需任何短训练示例。没有外部重置或老师强迫，我们的LSTM变体还学会了生成精确定时的尖峰和其他高度非线性周期性模式的稳定流。这使LSTM成为需要准确测量或生成时间间隔的任务的有前途的方法。
The temporal distance between events conveys information essential for numerous sequential tasks such as motor control and rhythm detection. While Hidden Markov Models tend to ignore this information, recurrent neural networks (RNNs) can in principle learn to make use of it. We focus on Long Short-Term Memory (LSTM) because it has been shown to outperform other RNNs on tasks involving long time lags. We find that LSTM augmented by "peephole connections" from its internal cells to its multiplicative gates can learn the fine distinction between sequences of spikes spaced either 50 or 49 time steps apart without the help of any short training exemplars. Without external resets or teacher forcing, our LSTM variant also learns to generate stable streams of precisely timed spikes and other highly nonlinear periodic patterns. This makes LSTM a promising approach for tasks that require the accurate measurement or generation of time intervals.