Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control
Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control
复制标题
DOI：
10.1109/tro.2021.3078317
复制
发表时间：
2021-05-26
影响因子：
7.8
通讯作者：
Huang, He
中科院分区：
文献类型：
--
作者：
Li, Minhan;Wen, Yue;Huang, He
个性化医疗设备（如下肢可穿戴机器人）具有挑战性。虽然膝关节假体控制参数调整过程自动化的初步可行性已经以原则性的方式得到了证明，但下一个关键问题是在临床环境中提高调整效率并为人类用户加速调整，同时保持人类安全。因此，我们提出了一个政策迭代与约束嵌入（PICE）方法作为一个创新的解决方案，在强化学习的框架下的问题。PICE的核心是使用一个投影贝尔曼方程的约束，保证政策评估过程中的性能值的正半定。此外，我们还开发了在线和离线PICE实现，为设计人员提供了额外的灵活性，可以充分利用测量数据，无论是从政策上还是政策外，以进一步提高PICE调整效率。我们的人类受试者测试表明，PICE提供了有效的政策，大大减少了调整时间。第一次，我们还通过将其应用于不同的任务和用户来实验性地评估和证明所部署的策略的鲁棒性。把它放在一起，我们的新的解决问题的方式已经有效的PICE已经证明了它的潜力，真正自动化的控制参数调整的机器人膝关节假体用户的过程。
Personalizing medical devices such as lower limb wearable robots is challenging. While the initial feasibility of automating the process of knee prosthesis control parameter tuning has been demonstrated in a principled way, the next critical issue is to improve tuning efficiency and speed it up for the human user, in clinic settings, while maintaining human safety. We, therefore, propose a policy iteration with constraint embedded (PICE) method as an innovative solution to the problem under the framework of reinforcement learning. Central to PICE is the use of a projected Bellman equation with a constraint of assuring positive semidefiniteness of performance values during policy evaluation. Additionally, we developed both online and offline PICE implementations that provide additional flexibility for the designer to fully utilize measurement data, either from on-policy or off-policy, to further improve PICE tuning efficiency. Our human subject testing showed that the PICE provided effective policies with significantly reduced tuning time. For the first time, we also experimentally evaluated and demonstrated the robustness of the deployed policies by applying them to different tasks and users. Putting it together, our new way of problem solving has been effective as PICE has demonstrated its potential toward truly automating the process of control parameter tuning for robotic knee prosthesis users.