Model-Free Reinforcement Learning with Ensemble for a Soft Continuum Robot Arm
Model-Free Reinforcement Learning with Ensemble for a Soft Continuum Robot Arm
复制标题
用于软连续体机器人手臂的无模型强化学习与集成
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Y. Kuniyoshi
中科院分区：
文献类型：
--
作者：
Ryota Morimoto;Satoshi Nishikawa;Ryuma Niiyama;Y. Kuniyoshi
软机器人比刚体机器人具有更多的被动自由度（DoF），这使得控制器设计变得困难。无模型强化学习（RL）是一种很有前途的工具，可以解决软机器人中的控制问题以及详细和精细的建模。然而，RL的软机器人的适应需要考虑柔软的身体的独特性质。在这项工作中，连续体机器人手臂被用作软机器人的一个例子，我们提出了一个集成的轻量级无模型强化学习网络（ELFNet），这是一个RL框架与计算轻合奏。我们证明了所提出的系统可以学习控制策略的连续体机器人手臂，以达到目标位置，不仅在模拟中，而且在真实的世界中使用它的提示。我们使用了一个可控制的连续机器人手臂，它由九块柔性橡胶人工肌肉组成。每个人造肌肉都可以通过压力控制阀独立控制，这表明可以单独使用真实的机器人来学习策略。我们发现，我们的方法比其他RL方法更适合于柔顺机器人，因为样本效率优于其他方法，并且当被动自由度的数量很大时，性能有显着差异。这项研究有望导致无模型RL在未来的软机器人控制的发展。
Soft robots have more passive degrees of freedom (DoFs) than rigid-body robots, which makes controller design difficult. Model-free reinforcement learning (RL) is a promising tool to resolve control problems in soft robotics alongside detailed and elaborate modeling. However, the adaptation of RL to soft robots requires consideration of the unique nature of soft bodies. In this work, a continuum robot arm is used as an example of a soft robot, and we propose an Ensembled Light-weight model-Free reinforcement learning Network (ELFNet), which is an RL framework with a computationally light ensemble. We demonstrated that the proposed system could learn control policies for a continuum robot arm to reach target positions using its tip not only in simulations but also in the real world. We used a pneumatically controlled continuum robot arm that operates with nine flexible rubber artificial muscles. Each artificial muscle can be controlled independently by pressure control valves, demonstrating that the policy can be learned using a real robot alone. We found that our method is more suitable for compliant robots than other RL methods because the sample efficiency is better than that of the other methods, and there is a significant difference in the performance when the number of passive DoFs is large. This study is expected to lead to the development of model-free RL in future soft robot control.