A Data-Driven Reinforcement Learning Solution Framework for Optimal and Adaptive Personalization of a Hip Exoskeleton
A Data-Driven Reinforcement Learning Solution Framework for Optimal and Adaptive Personalization of a Hip Exoskeleton
复制标题
DOI：
10.1109/icra48506.2021.9562062
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Xikai Tu;Minhan Li;Ming Liu;J. Si;H. Huang
中科院分区：
文献类型：
--
作者：
Xikai Tu;Minhan Li;Ming Liu;J. Si;H. Huang
机器人外骨骼是增强人类移动性的令人兴奋的技术。然而，设计这样的设备与人类用户无缝集成并辅助人类运动仍然是一个重大挑战。本文旨在开发一种新的基于强化学习（RL）的数据驱动解决方案框架，而无需首先对人-机器人动力学进行建模，以提供最佳和自适应的个性化扭矩辅助，从而减少人类在行走过程中的努力。我们的自动个性化解决方案框架包括辅助扭矩配置文件与两个控制定时参数（峰值和偏移时间），最小二乘策略迭代（LSPI）的学习参数调整政策，和成本函数的基础上转移的工作比率。所提出的控制器成功地验证了一个健康的人类受试者，以协助单侧髋关节伸展行走。结果表明，优化自适应RL控制器作为一种新的控制方法，能够有效地调节髋关节外骨骼的辅助力矩，使其与人体动作协调，降低人体髋关节伸肌的激活水平。
Robotic exoskeletons are exciting technologies for augmenting human mobility. However, designing such a device for seamless integration with the human user and to assist human movement still is a major challenge. This paper aims at developing a novel data-driven solution framework based on reinforcement learning (RL), without first modeling the human-robot dynamics, to provide optimal and adaptive personalized torque assistance for reducing human efforts during walking. Our automatic personalization solution framework includes the assistive torque profile with two control timing parameters (peak and offset timings), the least square policy iteration (LSPI) for learning the parameter tuning policy, and a cost function based on a transferred work ratio. The proposed controller was successfully validated on a healthy human subject to assist unilateral hip extension in walking. The results showed that the optimal and adaptive RL controller as a new approach was feasible for tuning assistive torque profile of the hip exoskeleton that coordinated with human actions and reduced activation level of hip extensor muscle in human.