Assist-as-needed robotic trainer based on reinforcement learning and its application to dart-throwing
Assist-as-needed robotic trainer based on reinforcement learning and its application to dart-throwing
复制标题
DOI：
10.1016/j.neunet.2014.01.012
复制
发表时间：
2014-05
期刊：
影响因子：
--
通讯作者：
Chihiro Obayashi;Tomoya Tamei;T. Shibata
中科院分区：
文献类型：
--
作者：
Chihiro Obayashi;Tomoya Tamei;T. Shibata
本文提出了一种用于运动技能学习的新型机器人训练器。它是用户自适应的，其灵感来自于物理治疗领域众所周知的“按需协助”原则。先前在机器人辅助运动技能学习领域的大多数研究都使用了预定的所需轨迹，并且尚未深入检查这些轨迹是否对每个用户来说都是最佳的。此外，指导假设指出，人类往往过于依赖外部辅助反馈，从而导致运动技能学习所需的内部反馈受到干扰。一些研究提出了一种根据用户表现调整辅助强度的系统，以防止用户过度依赖机器人辅助。然而，这些研究存在问题，因为需要用户运动系统的物理模型，而这本身就很难构建。在本文中，我们提出了一个用户自适应的机器人训练器框架，既不需要特定的所需轨迹，也不需要用户运动系统的物理模型，并且我们使用无模型强化学习来实现这一目标。我们选择投掷飞镖作为运动学习任务的示例，因为它是最简单的投掷任务之一，并且可以轻松定量测量其表现。对新手进行训练实验，旨在最大化飞镖得分并最小化物理机器人辅助，证明了所提出框架的可行性和合理性。
This paper proposes a novel robotic trainer for motor skill learning. It is user-adaptive inspired by the assist-as-needed principle well known in the field of physical therapy. Most previous studies in the field of the robotic assistance of motor skill learning have used predetermined desired trajectories, and it has not been examined intensively whether these trajectories were optimal for each user. Furthermore, the guidance hypothesis states that humans tend to rely too much on external assistive feedback, resulting in interference with the internal feedback necessary for motor skill learning. A few studies have proposed a system that adjusts its assistive strength according to the user’s performance in order to prevent the user from relying too much on the robotic assistance. There are, however, problems in these studies, in that a physical model of the user’s motor system is required, which is inherently difficult to construct. In this paper, we propose a framework for a robotic trainer that is user-adaptive and that neither requires a specific desired trajectory nor a physical model of the user’s motor system, and we achieve this using model-free reinforcement learning. We chose dart-throwing as an example motor-learning task as it is one of the simplest throwing tasks, and its performance can easily be and quantitatively measured. Training experiments with novices, aiming at maximizing the score with the darts and minimizing the physical robotic assistance, demonstrate the feasibility and plausibility of the proposed framework.