Reinforcement learning in multi-dimensional state-action space using random rectangular coarse coding and Gibbs sampling
Reinforcement learning in multi-dimensional state-action space using random rectangular coarse coding and Gibbs sampling
复制标题
DOI：
10.1109/iros.2007.4399401
复制
发表时间：
2007-12
期刊：
影响因子：
--
通讯作者：
K. Kimura
中科院分区：
文献类型：
--
作者：
K. Kimura
针对多维连续状态-动作空间中的强化学习问题，提出了一种基于粗编码技术和动作选择策略的强化学习方法。高维连续域上的强化学习包括两个问题：一是值函数逼近的推广问题，二是多维连续动作空间上动作选择的抽样问题。所提出的方法结合随机矩形粗编码的行动选择计划，使用吉布斯采样。随机矩形粗编码非常简单，非常适合于在高维空间中逼近Q函数和执行Gibbs采样。吉布斯采样使我们能够执行行动选择以下的Boltsmann分布在高维行动空间。通过Rod迷宫问题和冗余臂到达任务验证了该算法的有效性，并与传统的规则网格方法进行了比较。
This paper presents a coarse coding technique and an action selection scheme for reinforcement learning (RL) in multi-dimensional and continuous state-action spaces following conventional and sound RL manners. RL in high-dimensional continuous domains includes two issues: One is a generalization problem for value-function approximation, and the other is a sampling problem for action selection over multi-dimensional continuous action spaces. The proposed method combines random rectangular coarse coding with an action selection scheme using Gibbs-sampling. The random rectangular coarse coding is very simple and quite suited both to approximate Q-functions in high-dimensional spaces and to execute Gibbs sampling. Gibbs sampling enables us to execute action selection following Boltsmann distribution over high-dimensional action space. The algorithm is demonstrated through Rod in maze problem and a redundant-arm reaching task comparing with conventional regular grid approaches.