Human-Robotic Prosthesis as Collaborating Agents for Symmetrical Walking
Human-Robotic Prosthesis as Collaborating Agents for Symmetrical Walking
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Ruofan Wu;Junmin Zhong;Brent A. Wallace;Xiang Gao;H. Huang;Jennie Si
中科院分区：
文献类型：
--
作者：
Ruofan Wu;Junmin Zhong;Brent A. Wallace;Xiang Gao;H. Huang;Jennie Si
这是第一次尝试考虑人类的影响，在强化学习控制的机器人下肢假肢走向对称行走在真实的世界的情况。我们提出了一个协作多智能体强化学习（cMARL）解决方案框架，这个高度复杂和具有挑战性的人类假体协作（HPC）问题。HPC环境中的机器人的自动控制器的设计是基于已知会影响行走性能的可访问的物理特征或测量。比较与当前国家的最先进的机器人控制设计，这是基于单智能体，以及现有的MARL解决方案的方法量身定制的问题，包括多智能体深度确定性政策梯度（MADDPG）和反事实多智能体政策梯度（COMA）。结果表明，当这些方法相比，治疗的人和机器人作为耦合代理和使用估计的人类适应机器人控制设计可以实现较低的阶段成本，峰值误差，并改善对称性，以确保更好的人类行走性能。此外，我们的方法加速了步行任务的学习，提高了学习成功率。所提出的框架可以进一步发展，以研究人类和机器人下肢假肢如何相互作用，这是一个鲜为人知的领域。将cMARL推向真实的世界应用，例如用于规范步行的HPC，为人工智能如何对人们的生活产生积极影响树立了一个很好的榜样。
This is the first attempt at considering human influence in the reinforcement learning control of a robotic lower limb prosthesis toward symmetrical walking in real world situations. We propose a collaborative multi-agent reinforcement learning (cMARL) solution framework for this highly complex and challenging human-prosthesis collaboration (HPC) problem. The design of an automatic controller of the robot within the HPC context is based on accessible physical features or measurements that are known to affect walking performance. Comparisons are made with the current state-of-the-art robot control designs, which are single-agent based, as well as existing MARL solution approaches tailored to the problem, including multi-agent deep deterministic policy gradient (MADDPG) and counterfactual multi-agent policy gradient (COMA). Results show that, when compared to these approaches, treating the human and robot as coupled agents and using an estimated human adaption in robot control design can achieve lower stage cost, peak error, and improved symmetry to ensure better human walking performance. Additionally, our approach accelerates learning of walking tasks and increases learning success rate. The proposed framework can potentially be further developed to examine how human and robotic lower limb prosthesis interact, an area that little is known about. Advancing cMARL toward real world applications such as HPC for normative walking sets a good example of how AI can positively impact on people’s lives.