Shaping Individualized Impedance Landscapes for Gait Training via Reinforcement Learning
Shaping Individualized Impedance Landscapes for Gait Training via Reinforcement Learning
复制标题
DOI：
10.1109/tmrb.2021.3137971
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Yufeng Zhang;Shuai Li;Karen J. Nolan;D. Zanotto
中科院分区：
文献类型：
--
作者：
Yufeng Zhang;Shuai Li;Karen J. Nolan;D. Zanotto
按需协助(AAN)控制旨在通过鼓励患者的积极参与来促进机器人辅助康复的治疗结果。大多数AAN控制器使用阻抗控制来在目标运动周围创建顺应力场，以确保跟踪精度，同时允许适度的运动学误差。然而，由于控制力场形状的参数通常是基于对受试者学习能力的简单化假设而手动调整或在线调整的，传统的AAN控制器的有效性可能是有限的。在这项工作中，我们提出了一种新型的自适应人工神经网络控制器，它能够根据每个人的运动能力和任务要求，以一种依赖于相位的方式自主地重塑力场。该控制器包括改进的基于路径积分的策略改进算法、基于采样的无模型强化学习方法和分层策略参数评估结构，该方法通过指定性能驱动的学习目标来嵌入AAN范例。建议的控制策略对受试者运动反应的适应性及其促进短期运动适应的能力通过跑步机训练课程进行了实验验证，这些跑步机受试者在电动脚踝足矫形器的帮助下学习了改变的步态模式。
Assist-as-needed (AAN) control aims at promoting therapeutic outcomes in robot-assisted rehabilitation by encouraging patients’ active participation. Impedance control is used by most AAN controllers to create a compliant force field around a target motion to ensure tracking accuracy while allowing moderate kinematic errors. However, since the parameters governing the shape of the force field are often tuned manually or adapted online based on simplistic assumptions about subjects’ learning abilities, the effectiveness of conventional AAN controllers may be limited. In this work, we propose a novel adaptive AAN controller that is capable of autonomously reshaping the force field in a phase-dependent manner according to each individual’s motor abilities and task requirements. The proposed controller consists of a modified Policy Improvement with Path Integral algorithm, a model-free, sampling-based reinforcement learning method that learns a subject-specific impedance landscape in real-time, and a hierarchical policy parameter evaluation structure that embeds the AAN paradigm by specifying performance-driven learning goals. The adaptability of the proposed control strategy to subjects’ motor responses and its ability to promote short-term motor adaptations are experimentally validated through treadmill training sessions with able-bodied subjects who learned altered gait patterns with the assistance of a powered ankle-foot orthosis.