Adaptive Modular Reinforcement Learning for Robot Controlled in Multiple Environments
Adaptive Modular Reinforcement Learning for Robot Controlled in Multiple Environments
复制标题
多环境控制机器人的自适应模块化强化学习
DOI：
10.1109/access.2021.3070704
复制
发表时间：
2021-01-01
期刊：
影响因子：
3.9
通讯作者：
Shibuya, Takeshi
中科院分区：
文献类型：
--
作者：
Iwata, Teppei;Shibuya, Takeshi
针对多环境下的机器人控制问题，提出了一种自适应模块化强化学习体系结构和算法。强化学习通过智能体和被控系统之间的相互作用来自主获取控制规则。因此，强化学习有望应用于模型识别困难的机器人控制领域。这些机器人通常被期望在多种环境中操作。然而，现有的强化学习算法需要事先了解环境的变化。在本文中，我们提出了一种不需要预先了解环境的架构和算法。在这种体系结构中，可以根据与被控系统的交互来增加模块的数量，从而获得策略。因此，所提出的方法可以应用于动态变化的机器人，而不会失去强化学习算法不需要对被控系统的先验知识的特征。通过两个数值实验验证了该方法的有效性，与传统方法相比，该方法的性能提高了约25%。
This paper proposes an adaptive modular reinforcement learning architecture and an algorithm for robot control operating in multiple environments. Reinforcement learning autonomously acquires control rules by interacting between the agent and the controlled system. Consequently, reinforcement learning is expected to be applied to robot control where model identification is difficult. These robots are often expected to operate in multiple environments. However, existing reinforcement learning algorithms require prior knowledge of changes in the environment. In this paper, we proposed an architecture and algorithm that does not require prior knowledge of the environment. In this architecture, the policy can be acquired by increasing the number of modules based on the interaction with the controlled system. Therefore, the proposed method can be applied to robots whose dynamics change without losing the feature that the reinforcement learning algorithm does not require prior knowledge of the controlled system. Two numerical experiments were conducted to evaluate the proposed method, which improved the performance by approximately 25 % when compared to the conventional methods.