Generating Human Arm Kinematics Using Reinforcement Learning to Train Active Muscle Behavior in Automotive Research.
Generating Human Arm Kinematics Using Reinforcement Learning to Train Active Muscle Behavior in Automotive Research.
复制标题
在汽车研究中使用强化学习生成人体手臂运动学来训练主动肌肉行为。
DOI：
10.1115/1.4055680
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Panzer,MatthewB
中科院分区：
文献类型：
--
作者：
Mukherjee,Sayak;Perez-Rapela,Daniel;Forman,JasonL;Panzer,MatthewB
计算人体模型 (HBM) 是预测汽车碰撞环境下人体生物力学反应的重要工具。在许多情况下，通过将主动肌肉控制纳入 HBM 中，可以在不同的车辆操纵过程中生成逼真的运动学，从而改善对乘员反应的预测。在这项研究中，我们提出了一种基于强化学习（RL）开发主动肌肉控制器的方法。 RL 肌肉激活控制 (RL-MAC) 方法是对传统闭环反馈控制器的一种转变，它可以在控制器经过调整的有限负载条件范围内模拟准确的主动肌肉行为。相反，RL-MAC 使用迭代训练方法来产生主动肌肉力量以实现所需的关节运动，类似于儿童发展粗大运动技能的方式。在这项研究中，使用人体手臂的多体模型演示了深度确定性策略梯度 (DDPG) RL 控制器生成准确的人体运动学的能力。手臂模型经过训练，通过激活负责的肌肉来执行目标导向的肘部旋转，并使用两种招募方案进行研究：作为独立肌肉或作为拮抗肌群。经过训练的控制器的模拟表明，无论有或没有外部施加的负载，手臂都可以移动到目标位置。在恒定外部负载下训练的 RL-MAC 能够在简化的汽车碰撞场景下保持所需的肘关节角度，这意味着电机控制方法的稳健性。
Computational human body models (HBMs) are important tools for predicting human biomechanical responses under automotive crash environments. In many scenarios, the prediction of the occupant response will be improved by incorporating active muscle control into the HBMs to generate biofidelic kinematics during different vehicle maneuvers. In this study, we have proposed an approach to develop an active muscle controller based on reinforcement learning (RL). The RL muscle activation control (RL-MAC) approach is a shift from using traditional closed-loop feedback controllers, which can mimic accurate active muscle behavior under a limited range of loading conditions for which the controller has been tuned. Conversely, the RL-MAC uses an iterative training approach to generate active muscle forces for desired joint motion and is analogous to how a child develops gross motor skills. In this study, the ability of a deep deterministic policy gradient (DDPG) RL controller to generate accurate human kinematics is demonstrated using a multibody model of the human arm. The arm model was trained to perform goal-directed elbow rotation by activating the responsible muscles and investigated using two recruitment schemes: as independent muscles or as antagonistic muscle groups. Simulations with the trained controller show that the arm can move to the target position in the presence or absence of externally applied loads. The RL-MAC trained under constant external loads was able to maintain the desired elbow joint angle under a simplified automotive impact scenario, implying the robustness of the motor control approach.