Deep auto-encoder neural networks in reinforcement learning
Deep auto-encoder neural networks in reinforcement learning
复制标题
DOI：
10.1109/ijcnn.2010.5596468
复制
发表时间：
2010-07
期刊：
影响因子：
--
通讯作者：
S. Lange;Martin A. Riedmiller
中科院分区：
文献类型：
--
作者：
S. Lange;Martin A. Riedmiller
本文讨论了深度自动编码器神经网络在视觉强化学习（RL）任务中的有效性。我们提出了一个框架，用于将深度自动编码器的训练（用于学习紧凑的特征空间）与最近提出的批处理模式RL算法（用于学习策略）相结合。重点放在这种组合的数据效率，并研究由深度自动编码器自动构造的特征空间的属性。这些特征空间的经验表明，充分类似于现有的相似性和观测之间的空间关系，并允许学习有用的政策。我们提出了几种方法，利用任务相关的信息来改善特征空间的拓扑结构。最后，我们提出了第一个成功地学习良好的控制策略，直接对合成和真实的图像的结果。
This paper discusses the effectiveness of deep auto-encoder neural networks in visual reinforcement learning (RL) tasks. We propose a framework for combining the training of deep auto-encoders (for learning compact feature spaces) with recently-proposed batch-mode RL algorithms (for learning policies). An emphasis is put on the data-efficiency of this combination and on studying the properties of the feature spaces automatically constructed by the deep auto-encoders. These feature spaces are empirically shown to adequately resemble existing similarities and spatial relations between observations and allow to learn useful policies. We propose several methods for improving the topology of the feature spaces making use of task-dependent information. Finally, we present first results on successfully learning good control policies directly on synthesized and real images.