Collaborative Research: Reinforcement learning based adaptive optimal control of powered knee prosthesis for human users in real life

协作研究：基于强化学习的现实生活中人类用户动力膝关节假体的自适应最优控制

• 批准号: 1808752
• 负责人: Jennie Si
• 金额: $ 25.09万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1808752&HistoricalAwards=false
• 关键词: Collaborative; Research; Reinforcement; learning; based

The proposed research aims at designing robust, real time learning controllers for powered lower limb prosthesis worn by above-knee amputees. It centers on adaptive optimal tuning of prosthetic knee joint impedance parameters with an ultimate goal of achieving human-prosthesis symbiosis. Current state-of-the-art approaches rely on a predetermined collection of knee joint impedance parameters, resulted from tedious manual tuning in a clinic. In addition to a lack of adaptability to different users, current impedance controls do not adapt to different use environments. One of the key design challenge is due to the constant interaction between the human user and the robotic leg. As such, advanced robotics including those employing latest artificial intelligence technologies, control system theory and design, and existing biomechanics based controls cannot meet the needs of real time learning control of a powered prosthetic leg in a human-prosthesis system. Given the nature of the problem, reinforcement learning based adaptive optimal control, also referred to as adaptive dynamic programming (ADP), holds great promise to delivering the next generation of prosthesis control solutions. Intellectual Merit: The design challenge requires innovative approaches of real time reinforcement learning control. The learning controller has to be designed without knowing an explicit dynamic system model describing the human-prosthesis system, while assuring human user safety and system stability, and being scalable and adaptable to different users and use conditions. Putting it all together, the success of this project will be an important milestone for machine learning, control engineering, and rehabilitation engineering. Broader Impacts: This research has a direct impact on improving the lives of above-knee amputees. Also of great societal impact is the potential of reducing health care cost. New knowledge gained from human-robot interaction will not only aid amputees but also stroke patients who use exoskeleton as assistive devices. The proposed research will also benefit several research communities such as wearable robots, machine learning, and rehabilitation to develop new technologies addressing real applications. To excite and educate future leaders and researchers in science and engineering, the project will provide an opportunity for integration of our research work into graduate education and postdoc training.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

提出的研究旨在为膝部以上截肢者佩戴的动力下肢假体设计鲁棒的实时学习控制器。其核心是对假体膝关节阻抗参数的自适应优化调整，最终目标是实现人与假体的共生。目前最先进的方法依赖于预先收集的膝关节阻抗参数，这是在诊所中繁琐的手动调整造成的。除了缺乏对不同用户的适应性外，电流阻抗控制不能适应不同的使用环境。其中一个关键的设计挑战是由于人类用户和机器人腿之间的不断互动。因此，先进的机器人技术，包括采用最新的人工智能技术，控制系统理论和设计，以及现有的基于生物力学的控制，都不能满足人体假肢系统中动力假肢的实时学习控制需求。考虑到问题的本质，基于强化学习的自适应最优控制，也被称为自适应动态规划（ADP），有望提供下一代假肢控制解决方案。智力优势：设计挑战需要实时强化学习控制的创新方法。学习控制器必须在不知道描述人体-假肢系统的显式动态系统模型的情况下进行设计，同时保证人类用户的安全性和系统的稳定性，并具有可扩展性和适应性，以适应不同的用户和使用条件。综上所述，这个项目的成功将是机器学习、控制工程和康复工程的一个重要里程碑。更广泛的影响：这项研究对改善膝盖以上截肢者的生活有直接的影响。降低医疗保健费用的潜力也具有巨大的社会影响。从人机交互中获得的新知识不仅可以帮助截肢者，还可以帮助使用外骨骼作为辅助设备的中风患者。拟议的研究还将使几个研究社区受益，如可穿戴机器人、机器学习和康复，以开发解决实际应用的新技术。为了激发和培养未来的科学和工程领域的领导者和研究人员，该项目将为我们的研究工作提供一个整合研究生教育和博士后培训的机会。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Online Reinforcement Learning Control by Direct Heuristic Dynamic Programming: From Time-Driven to Event-Driven

• DOI: 10.1109/tnnls.2021.3053037
• 发表时间: 2020-06
• 期刊: IEEE Transactions on Neural Networks and Learning Systems
• 影响因子: 10.4
• 作者: Qingtao Zhao;J. Si;Jian Sun-

Robotic Knee Tracking Control to Mimic the Intact Human Knee Profile Based on Actor-Critic Reinforcement Learning

• DOI: 10.1109/jas.2021.1004272
• 发表时间: 2022-01-01
• 期刊: IEEE-CAA JOURNAL OF AUTOMATICA SINICA
• 影响因子: 11.8
• 作者: Wu, Ruofan;Yao, Zhikai;Huang, He Helen
• 通讯作者: Huang, He Helen

Online Reinforcement Learning Control for the Personalization of a Robotic Knee Prosthesis

• DOI: 10.1109/tcyb.2019.2890974
• 发表时间: 2020-06-01
• 期刊: IEEE TRANSACTIONS ON CYBERNETICS
• 影响因子: 11.8
• 作者: Wen, Yue;Si, Jennie;Huang, He (Helen)
• 通讯作者: Huang, He (Helen)

Toward Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control

• DOI: 10.1109/tro.2021.3078317
• 发表时间: 2021-05-26
• 期刊: IEEE TRANSACTIONS ON ROBOTICS
• 影响因子: 7.8
• 作者: Li, Minhan;Wen, Yue;Huang, He
• 通讯作者: Huang, He

Taking both sides: seeking symbiosis between intelligent prostheses and human motor control during locomotion

• DOI: 10.1016/j.cobme.2021.100314
• 发表时间: 2021-07-24
• 期刊: CURRENT OPINION IN BIOMEDICAL ENGINEERING
• 影响因子: 3.9
• 作者: Huang, He (Helen);Si, Jennie;Li, Minhan
• 通讯作者: Li, Minhan