INTELLIGENT CONTROL OF UPPER EXTREMITY NEURAL PROSTHESES

上肢神经假体的智能控制 脊髓损伤(SCI)和神经运动障碍的一个共同特征是周围神经肌肉系统保持完整。功能性电刺激(FES)为这些个体提供了恢复运动的潜力。最近在电极和传感器硬件方面取得了令人印象深刻的改进，但开发复杂动态运动的控制算法仍然很困难。 强化学习(RL)是人工智能的一种技术，具有克服这一问题的潜力。基于RL的控制系统从经验中学习如何控制运动，就像婴儿一样。该系统从多个传感器接收信息以及奖励信号，并产生动作，即肌肉刺激水平，这些动作最初是随机的。该系统将学会预测其行为的后果，并最终收敛到一种控制策略，使回报之和随着时间的推移最大化。RL的一个基本特点是控制策略不是设计者创造的，而是从经验中学习的。这种学习过程最终可能导致比传统设计的反馈控制系统更高质量的运动行为，这些反馈控制系统倾向于与身体的自然动力学(如惯性、摆和质量-弹簧机构)进行斗争，而不是利用它们。此外，自学习系统的优点是它可以适应使用者的身体质量、肌肉力量以及电极位置的变化。 长期目标是一个系统，将来自用户的高级命令与来自植入的传感器的信号相结合，以产生智能和自适应的运动功能。这一概念的可行性将在这里测试FES对上肢六块肌肉的控制，以执行在水平面上伸展的任务。提出了以下具体目标：(1)利用计算机生成的命令和奖励在虚拟手臂上实现RL控制；(2)通过操作员给出的命令和奖励在虚拟手臂上实现RL控制；以及(3)通过用户通过基于头部跟踪器的输入设备给出的命令和奖励来控制两个高位颈髓损伤受试者瘫痪手臂中的肌肉。

INTELLIGENT CONTROL OF UPPER EXTREMITY NEURAL PROSTHESES A common feature of spinal cord injury (SCI) and neurological movement disorders is that the peripheral neuromuscular system remains intact. Functional Electrical Stimulation (FES) offers the potential to restore movement in these individuals. Impressive improvements in electrode and sensor hardware have recently been made, but development of control algorithms for complex dynamic movements remains difficult. Reinforcement learning (RL) is a technique from artificial intelligence that has the potential to overcome this problem. A RL-based control system learns from experience how to control movement, in very much the same way as an infant. The system receives information from multiple sensors, as well as a reward signal, and generates actions, i.e. muscle stiimulation levels, that are initially random. The system will learn to predict the consequences of its actions and will ultimately converge to a control strategy that maximizes the sum of rewards over time. An essential feature of RL is that the control strategy is not created by the designer, but is learned from experience. This learning process could ultimately result in motor behavior of much higher quality than can be achieved with traditionally designed feedback control systems, which tend to "fight" rather than exploit the natural dynamics of the body such as inertia, pendulum and mass-spring mechanisms. Furthermore, a self learning system has the advantage that it can adapt itself to the user's body mass, muscle strength, as well as variations in electrode location. The long-term goal is a system that integrates high-level commands from the user with signals from implanted sensors to produce intelligent and adaptive motor function. Feasibility of this concept will be tested here for FES control of six muscles in the upper extremity, to perform the task of reaching in the horizontal plane. The following specific aims are proposed: (1) Implementation of RL control on a virtual arm with computer-generated commands and rewards, (2) RL control on a virtual arm, with commands and rewards given by a human operator, and (3) RL control of muscles in a paralyzed arm in two subjects with high cervical spinal cord injury, with commands and rewards given by the user via a head tracker based input device.