Fast Reinforcement Learning for Three-Dimensional Kinetic Human–Robot Cooperation with an EMG-to-Activation Model
Fast Reinforcement Learning for Three-Dimensional Kinetic Human–Robot Cooperation with an EMG-to-Activation Model
复制标题
DOI：
10.1163/016918611x558252
复制
发表时间：
2011-01
影响因子：
2
通讯作者：
Tomoya Tamei;T. Shibata
中科院分区：
文献类型：
--
作者：
Tomoya Tamei;T. Shibata
动力学人机合作已在动力辅助和康复等研究领域进行了研究。肌电图 (EMG) 信号通常用于此目的，而不是力传感器，因为肌电图信号反映了用户的运动意图，在实际运动之前观察到，从而导致更自然的交互。然而，使用 EMG 信号的一个固有问题是它们的时变特性，这是由于校准阶段和实际任务中的闭环反馈系统之间的差异导致肌肉协调会随着时间的推移而变化。本文提出使用策略梯度类型的强化学习来克服这个问题，将基于肌电图的动态人机合作任务制定为目标导向任务，其中估计用户用于动态交互所施加的力以实现与机器人共享的目标。强化学习使力估计器能够适应肌电图信号的时变特性。力估计器基于所谓的肌电图激活模型，该模型在生物学上是合理的，并且只有少量参数，可以快速学习。三维合作转移任务证明了我们方法的可行性。
Kinetic human–machine cooperation has been investigated in research fields such as power assist and rehabilitation. Electromyographic (EMG) signals have often been used for this purpose instead of force sensors, since the EMG signals reflect the motor intention of a user, observed prior to actual movements, leading to more natural interaction. However, an inherent problem in using EMG signals is their time-varying nature caused by the fact that muscle coordination can vary over time because of differences between the closed-loop feedback systems in the calibration stage and in the actual task. This paper proposes the use of a policy gradient type of reinforcement learning for overcoming this problem by formulating EMG-based kinetic human–robot cooperative tasks as goal-oriented tasks, in which the force exerted by the user for kinetic interaction is estimated to achieve a goal shared with the robot. The reinforcement learning enables the force estimator to be adaptive to the time-varying nature of the EMG signals. The force estimator is based on the so-called EMG-to-activation model which is biologically plausible and has only a small number of parameters, enabling fast learning. A three-dimensional cooperative transfer task demonstrates the feasibility of our approach.