Using Reinforcement Learning to Estimate Human Joint Moments From Electromyography or Joint Kinematics: An Alternative Solution to Musculoskeletal-Based Biomechanics
Using Reinforcement Learning to Estimate Human Joint Moments From Electromyography or Joint Kinematics: An Alternative Solution to Musculoskeletal-Based Biomechanics
复制标题
使用强化学习根据肌电图或关节运动学估计人体关节力矩：基于肌肉骨骼的生物力学的替代解决方案
DOI：
10.1115/1.4049333
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Huang, He
中科院分区：
文献类型：
--
作者：
Wu, Wen;Saul, Katherine R.;Huang, He
强化学习（RL）有可能为运动分析中估计关节力矩的现有挑战提供创新的解决方案，例如运动学或肌电图（EMG）噪声和未知的模型参数。在这里，我们探讨RL的可行性，以协助生物力学应用的关节力矩估计。采集了6名健康受试者手指和手腕自由运动时前臂和手的运动学和前臂四块肌肉的肌电图。使用邻近策略优化方法，我们训练了两种类型的RL代理，分别基于测量的运动学或测量的EMG来估计关节力矩。为了量化经过训练的RL代理的性能，使用估计的关节力矩来驱动用于估计运动学的前向动力学模型，然后使用Pearson相关系数将其与测量的运动学进行比较。结果表明，这两个训练的RL代理是可行的，以估计手腕和掌指关节（MCP）运动预测的关节力矩。预测和测量的运动学之间的相关系数，来自运动学驱动的代理和受试者特定的EMG驱动的代理，分别为98% ± 1%和94% ± 3%的手腕，分别为95% ± 2%和84% ± 6%的掌指关节，分别。此外，生物力学上合理的关节力矩-角度-EMG关系（即，关节力矩对关节角度和EMG的依赖性）仅使用15秒收集的数据来预测。总之，这项研究表明，RL方法可以是一种替代技术，以传统的逆动力学分析在人体生物力学研究和肌电驱动的人机接口应用。
Reinforcement learning (RL) has potential to provide innovative solutions to existing challenges in estimating joint moments in motion analysis, such as kinematic or electromyography (EMG) noise and unknown model parameters. Here, we explore feasibility of RL to assist joint moment estimation for biomechanical applications. Forearm and hand kinematics and forearm EMGs from four muscles during free finger and wrist movement were collected from six healthy subjects. Using the proximal policy optimization approach, we trained two types of RL agents that estimated joint moment based on measured kinematics or measured EMGs, respectively. To quantify the performance of trained RL agents, the estimated joint moment was used to drive a forward dynamic model for estimating kinematics, which was then compared with measured kinematics using Pearson correlation coefficient. The results demonstrated that both trained RL agents are feasible to estimate joint moment for wrist and metacarpophalangeal (MCP) joint motion prediction. The correlation coefficients between predicted and measured kinematics, derived from the kinematics-driven agent and subject-specific EMG-driven agents, were 98% ± 1% and 94% ± 3% for the wrist, respectively, and were 95% ± 2% and 84% ± 6% for the metacarpophalangeal joint, respectively. In addition, a biomechanically reasonable joint moment-angle-EMG relationship (i.e., dependence of joint moment on joint angle and EMG) was predicted using only 15 s of collected data. In conclusion, this study illustrates that an RL approach can be an alternative technique to conventional inverse dynamic analysis in human biomechanics study and EMG-driven human-machine interfacing applications.
登录
查看更多内容
DOI：
10.1115/1.2834295
发表时间：
1998-02-01
影响因子：
1.7
作者：
Kuo, AD
通讯作者：
Kuo, AD
影响因子：
--
作者：
F. Zajac
通讯作者：
F. Zajac
DOI：
10.1682/jrrd.2010.09.0177
发表时间：
2011-01-01
影响因子：
--
作者：
Scheme, Erik;Englehart, Kevin
通讯作者：
Englehart, Kevin
影响因子：
2.5
作者：
Formica, Domenico;Charles, Steven K.;Krebs, Hermano I.
通讯作者：
Krebs, Hermano I.
评估肩部肌肉和关节功能的受试者特定肌肉骨骼模型。
影响因子：
2.4
作者：
Wen Wu;P. Lee;A. Bryant;M. Galea;D. Ackland
通讯作者：
D. Ackland