Adaptive balancing of exploration and exploitation around the edge of chaos in internal-chaos-based learning
Adaptive balancing of exploration and exploitation around the edge of chaos in internal-chaos-based learning
复制标题
DOI：
10.1016/j.neunet.2020.08.002
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
Toshitaka Matsuki;K. Shibata
中科院分区：
文献类型：
--
作者：
Toshitaka Matsuki;K. Shibata
本文讨论了由神经网络混沌内部动力学驱动的探索学习。Hoerzer等研究表明，混沌储层网络（RN）可以在外部随机噪声和顺序奖励驱动下进行探索学习。在本文中，我们证明了混沌RN可以在没有外部噪声的情况下学习，因为源自其内部混沌动力学的输出波动起到了探索的作用。随着学习的进行，混沌性逐渐减小，网络可以自动从探索模式切换到开发模式。此外，当遇到新的情况时，网络可以恢复探索。此外，我们发现，即使影响混沌性的两个参数不同，学习性能也总是在混沌边缘附近提高。从这些结果来看，我们认为探索是由内部混沌动力学产生的，而开发则是在混沌动力学上通过学习形成吸引子的过程中出现的。因此，探索和利用在混沌边缘得到了很好的平衡，从而产生了良好的学习性能。
This paper addresses learning with exploration driven by chaotic internal dynamics of a neural network. Hoerzer et al. showed that a chaotic reservoir network (RN) can learn with exploration driven by external random noise and a sequential reward. In this paper, we demonstrate that a chaotic RN can learn without external noise because the output fluctuation originated from its internal chaotic dynamics functions as exploration. As learning progresses, the chaoticity decreases and the network can automatically switch from exploration mode to exploitation mode. Furthermore, the network can resume exploration when presented with a new situation. In addition, we found that even when the two parameters that influence the chaoticity are varied, learning performance always improves around the edge of chaos. From these results, we think that exploration is generated from internal chaotic dynamics, and exploitation appears in the process of forming attractors on the chaotic dynamics through learning. Consequently, exploration and exploitation are well-balanced around the edge of chaos, which leads to good learning performance.