Collaborative Research: Reinforcement learning based adaptive optimal control of powered knee prosthesis for human users in real life

建议的研究旨在设计强大的，真实的时间学习控制器的动力下肢假肢穿膝上截肢者。它集中在假体膝关节阻抗参数的自适应优化调整，最终目标是实现人体假体共生。目前最先进的方法依赖于膝关节阻抗参数的预定集合，这是由诊所中繁琐的手动调谐产生的。除了缺乏对不同用户的适应性之外，当前的阻抗控制不适应不同的使用环境。其中一个关键的设计挑战是由于人类用户和机器人腿之间的不断互动。因此，包括采用最新人工智能技术、控制系统理论和设计以及现有的基于生物力学的控制的先进机器人技术不能满足人类假肢系统中的动力假肢腿的真实的时间学习控制的需要。鉴于问题的性质，基于强化学习的自适应最优控制，也称为自适应动态规划（ADP），有望提供下一代假肢控制解决方案。智力优势：设计挑战需要创新的真实的时间强化学习控制方法。学习控制器必须在不知道描述人体假体系统的显式动态系统模型的情况下设计，同时确保人类用户的安全性和系统稳定性，并且可扩展和适应不同的用户和使用条件。总而言之，该项目的成功将成为机器学习、控制工程和康复工程的重要里程碑。更广泛的影响：这项研究对改善膝上截肢者的生活有直接影响。降低医疗保健成本的潜力也具有巨大的社会影响。从人机交互中获得的新知识不仅可以帮助截肢者，还可以帮助使用外骨骼作为辅助设备的中风患者。拟议的研究还将使可穿戴机器人、机器学习和康复等多个研究领域受益，以开发解决真实的应用的新技术。为了激励和教育未来的科学和工程领导者和研究人员，该项目将为我们的研究工作融入研究生教育和博士后培训提供一个机会。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The proposed research aims at designing robust, real time learning controllers for powered lower limb prosthesis worn by above-knee amputees. It centers on adaptive optimal tuning of prosthetic knee joint impedance parameters with an ultimate goal of achieving human-prosthesis symbiosis. Current state-of-the-art approaches rely on a predetermined collection of knee joint impedance parameters, resulted from tedious manual tuning in a clinic. In addition to a lack of adaptability to different users, current impedance controls do not adapt to different use environments. One of the key design challenge is due to the constant interaction between the human user and the robotic leg. As such, advanced robotics including those employing latest artificial intelligence technologies, control system theory and design, and existing biomechanics based controls cannot meet the needs of real time learning control of a powered prosthetic leg in a human-prosthesis system. Given the nature of the problem, reinforcement learning based adaptive optimal control, also referred to as adaptive dynamic programming (ADP), holds great promise to delivering the next generation of prosthesis control solutions. Intellectual Merit: The design challenge requires innovative approaches of real time reinforcement learning control. The learning controller has to be designed without knowing an explicit dynamic system model describing the human-prosthesis system, while assuring human user safety and system stability, and being scalable and adaptable to different users and use conditions. Putting it all together, the success of this project will be an important milestone for machine learning, control engineering, and rehabilitation engineering. Broader Impacts: This research has a direct impact on improving the lives of above-knee amputees. Also of great societal impact is the potential of reducing health care cost. New knowledge gained from human-robot interaction will not only aid amputees but also stroke patients who use exoskeleton as assistive devices. The proposed research will also benefit several research communities such as wearable robots, machine learning, and rehabilitation to develop new technologies addressing real applications. To excite and educate future leaders and researchers in science and engineering, the project will provide an opportunity for integration of our research work into graduate education and postdoc training.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.