Collaborative Research: Reinforcement learning based adaptive optimal control of powered knee prosthesis for human users in real life

这项研究旨在为膝上截肢者佩戴的电动假肢设计健壮的实时学习控制器。它的核心是假体膝关节阻抗参数的自适应优化调整，最终目标是实现人-假体共生。目前最先进的方法依赖于预先确定的膝关节阻抗参数集合，这是由诊所繁琐的手动调整产生的。除了缺乏对不同用户的适应性外，电流阻抗控制也不适应不同的使用环境。其中一个关键的设计挑战是由于人类用户和机械腿之间的持续交互。因此，先进的机器人技术，包括采用最新人工智能技术、控制系统理论和设计的机器人，以及现有的基于生物力学的控制，不能满足人类假肢系统中动力假肢的实时学习控制的需要。鉴于问题的本质，基于强化学习的自适应最优控制，也被称为自适应动态规划(ADP)，为提供下一代假肢控制解决方案带来了巨大的希望。智力优势：设计挑战需要实时强化学习控制的创新方法。学习控制器必须在不知道描述人体-假肢系统的显式动态系统模型的情况下设计，同时确保人类用户的安全和系统的稳定性，并且可扩展和适应不同的用户和使用条件。综上所述，该项目的成功将是机器学习、控制工程和康复工程的一个重要里程碑。更广泛的影响：这项研究对改善膝盖以上截肢者的生活有直接影响。降低医疗费用的潜力也具有重大的社会影响。从人与机器人互动中获得的新知识不仅将帮助截肢者，也将帮助使用外骨骼作为辅助设备的中风患者。拟议的研究还将使可穿戴机器人、机器学习和康复等几个研究社区受益，以开发解决实际应用的新技术。为了激励和教育未来的科学和工程领域的领导者和研究人员，该项目将提供一个机会，将我们的研究工作整合到研究生教育和博士后培训中。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The proposed research aims at designing robust, real time learning controllers for powered lower limb prosthesis worn by above-knee amputees. It centers on adaptive optimal tuning of prosthetic knee joint impedance parameters with an ultimate goal of achieving human-prosthesis symbiosis. Current state-of-the-art approaches rely on a predetermined collection of knee joint impedance parameters, resulted from tedious manual tuning in a clinic. In addition to a lack of adaptability to different users, current impedance controls do not adapt to different use environments. One of the key design challenge is due to the constant interaction between the human user and the robotic leg. As such, advanced robotics including those employing latest artificial intelligence technologies, control system theory and design, and existing biomechanics based controls cannot meet the needs of real time learning control of a powered prosthetic leg in a human-prosthesis system. Given the nature of the problem, reinforcement learning based adaptive optimal control, also referred to as adaptive dynamic programming (ADP), holds great promise to delivering the next generation of prosthesis control solutions. Intellectual Merit: The design challenge requires innovative approaches of real time reinforcement learning control. The learning controller has to be designed without knowing an explicit dynamic system model describing the human-prosthesis system, while assuring human user safety and system stability, and being scalable and adaptable to different users and use conditions. Putting it all together, the success of this project will be an important milestone for machine learning, control engineering, and rehabilitation engineering. Broader Impacts: This research has a direct impact on improving the lives of above-knee amputees. Also of great societal impact is the potential of reducing health care cost. New knowledge gained from human-robot interaction will not only aid amputees but also stroke patients who use exoskeleton as assistive devices. The proposed research will also benefit several research communities such as wearable robots, machine learning, and rehabilitation to develop new technologies addressing real applications. To excite and educate future leaders and researchers in science and engineering, the project will provide an opportunity for integration of our research work into graduate education and postdoc training.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.