A study of reinforcement learning for the robot with many degrees of freedom - acquisition of locomotion patterns for multi-legged robot
A study of reinforcement learning for the robot with many degrees of freedom - acquisition of locomotion patterns for multi-legged robot
复制标题
多自由度机器人强化学习研究——多足机器人运动模式获取
DOI：
10.1109/robot.2002.1014235
复制
发表时间：
2002
期刊：
影响因子：
--
通讯作者：
F. Matsuno
中科院分区：
文献类型：
--
作者：
Kazuyuki Ito;F. Matsuno
强化学习最近作为一种不仅适用于玩具问题而且适用于复杂系统（例如机器人系统）的学习方法而受到广泛关注。它不需要先验知识，具有较高的反应和适应行为能力。然而，动作状态空间的增加使得学习过程难以完成。在之前的大多数工作中，学习的应用仅限于具有较小动作状态空间的简单任务。考虑到这一点，我们提出了一种新的强化学习算法：基于遗传算法的动态构建探索空间的Q学习。该算法适用于具有高维动作和内部状态空间的系统，例如具有许多冗余自由度的机器人。为了证明所提出算法的有效性，对 12 足机器人的运动模式进行了模拟。结果，使用我们提出的算法获得了有效的行为。
Reinforcement learning has recently been receiving much attention as a learning method for not only toy problems but also complicated systems such as robot systems. It does not need priori knowledge and has higher capability of reactive and adaptive behaviors. However, increasing of action-state space makes it difficult to accomplish the learning process. In most of the previous works, the application of the learning is restricted to simple tasks with a small action-state space. Considering this point, we present a new reinforcement learning algorithm: Q-learning with dynamic structuring of exploration space based on genetic algorithm. The algorithm is applicable to systems with high dimensional action and interior state spaces, for example, a robot with many redundant degrees of freedom. To demonstrate the effectiveness of the proposed algorithm simulations of locomotion patterns for a 12-leged robot were carried out. As the result, an effective behavior was obtained by using our proposed algorithm.