Biological arm motion through reinforcement learning
Biological arm motion through reinforcement learning
复制标题
DOI：
10.1007/s00422-004-0485-3
复制
发表时间：
2004-07-01
影响因子：
1.9
通讯作者：
Ito, K
中科院分区：
文献类型：
--
作者：
Izawa, J;Kondo, T;Ito, K
针对具有冗余执行器的生物系统，提出了一种基于强化学习的最优学习控制方法。由于肌肉激活空间的冗余性，使得强化学习很难应用于生物控制系统。我们用下面的方法解决这个问题。首先，我们将控制输入空间按照学习的优先级划分为两个子空间，并将强化学习的搜索噪声限制在第一优先级子空间。然后，随着学习的进行，约束减少，搜索空间扩展到第二优先级子空间。高优先级子空间被设计成使得臂的阻抗可以是高的。通过强化学习获得平滑的到达运动，而无需任何先前的手臂动力学知识。
The present paper discusses an optimal learning control method using reinforcement learning for biological systems with a redundant actuator. It is difficult to apply reinforcement learning to biological control systems because of the redundancy in muscle activation space. We solve this problem with the following method. First, we divide the control input space into two subspaces according to a priority order of learning and restrict the search noise for reinforcement learning to the first priority subspace. Then the constraint is reduced as the learning progresses, with the search space extending to the second priority subspace. The higher priority subspace is designed so that the impedance of the arm can be high. A smooth reaching motion is obtained through reinforcement learning without any previous knowledge of the arm's dynamics.