Reinforcement Learning of a Memory Task Using an Echo State Network with Multi-layer Readout
Reinforcement Learning of a Memory Task Using an Echo State Network with Multi-layer Readout
复制标题
使用具有多层读出的回声状态网络对记忆任务进行强化学习
DOI：
10.1007/978-3-319-78452-6_2
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Shibata Katsunari
中科院分区：
文献类型：
--
作者：
Matsuki Toshitaka;Shibata Katsunari
通过强化学习（RL）训练神经网络（NN）最近受到关注，循环神经网络（RNN）用于需要记忆的学习任务。同时，为了弥补学习 RNN 的缺点，储层网络（RN）通常主要用于监督学习。 RN是一种特殊的RNN，由于其丰富的动态表示而备受关注。研究了一种涉及使用多层读出 (MLR)（包括多层神经网络）的方法，用于使用 RN 获取复杂表示。这项研究表明，具有 MLR 的 RN 可以通过 RL 和反向传播来学习“记忆任务”。此外，完成任务所需的非线性表示在 RN 中不会被观察到，而是通过在 MLR 中学习来构建。结果表明 MLR 可以弥补 RN 中有限的计算能力。
Training a neural network (NN) through reinforcement learning (RL) has been focused on recently, and a recurrent NN (RNN) is used in learning tasks that require memory. Meanwhile, to cover the shortcomings in learning an RNN, the reservoir network (RN) has been often employed mainly in supervised learning. The RN is a special RNN and has attracted much attention owing to its rich dynamic representations. An approach involving the use of a multi-layer readout (MLR), which comprises a multi-layer NN, was studied for acquiring complex representations using the RN. This study demonstrates that an RN with MLR can learn a “memory task” through RL with back propagation. In addition, non-linear representations required to clear the task are not observed in the RN but are constructed by learning in the MLR. The results suggest that the MLR can make up for the limited computational ability in an RN.