INTELLIGENT CONTROL OF UPPER EXTREMITY NEURAL PROSTHESES

上肢神经假体的智能控制 脊髓损伤（SCI）和神经运动障碍的一个共同特征是周围神经肌肉系统保持完整。功能性电刺激（FES）提供了恢复这些人运动的可能性。最近在电极和传感器硬件方面取得了令人印象深刻的改进，但复杂动态运动的控制算法的开发仍然很困难。 强化学习（RL）是一种来自人工智能的技术，有可能克服这个问题。基于RL的控制系统从经验中学习如何控制运动，就像婴儿一样。该系统接收来自多个传感器的信息以及奖励信号，并生成最初随机的动作，即肌肉刺激水平。系统将学习预测其行为的后果，并最终收敛到一个控制策略，随着时间的推移，最大化奖励的总和。RL的一个基本特征是控制策略不是由设计者创建的，而是从经验中学习的。这种学习过程最终可能导致比传统设计的反馈控制系统更高质量的运动行为，传统设计的反馈控制系统倾向于“对抗”而不是利用身体的自然动力学，如惯性，钟摆和质量弹簧机制。此外，自学习系统的优点在于，它可以使自身适应于用户的体重、肌肉强度以及电极位置的变化。 长期目标是一个系统，它将来自用户的高级命令与来自植入传感器的信号集成在一起，以产生智能和自适应的运动功能。这一概念的可行性将在这里测试的FES控制的六个肌肉在上肢，执行的任务，达到在水平面。提出了以下具体目标：（1）利用计算机生成的命令和奖励在虚拟手臂上实现RL控制，（2）利用人类操作员给出的命令和奖励在虚拟手臂上实现RL控制，以及（3）利用用户经由基于头部跟踪器的输入设备给出的命令和奖励在两个具有高颈脊髓损伤的受试者中瘫痪手臂中的肌肉的RL控制。

INTELLIGENT CONTROL OF UPPER EXTREMITY NEURAL PROSTHESES A common feature of spinal cord injury (SCI) and neurological movement disorders is that the peripheral neuromuscular system remains intact. Functional Electrical Stimulation (FES) offers the potential to restore movement in these individuals. Impressive improvements in electrode and sensor hardware have recently been made, but development of control algorithms for complex dynamic movements remains difficult. Reinforcement learning (RL) is a technique from artificial intelligence that has the potential to overcome this problem. A RL-based control system learns from experience how to control movement, in very much the same way as an infant. The system receives information from multiple sensors, as well as a reward signal, and generates actions, i.e. muscle stiimulation levels, that are initially random. The system will learn to predict the consequences of its actions and will ultimately converge to a control strategy that maximizes the sum of rewards over time. An essential feature of RL is that the control strategy is not created by the designer, but is learned from experience. This learning process could ultimately result in motor behavior of much higher quality than can be achieved with traditionally designed feedback control systems, which tend to "fight" rather than exploit the natural dynamics of the body such as inertia, pendulum and mass-spring mechanisms. Furthermore, a self learning system has the advantage that it can adapt itself to the user's body mass, muscle strength, as well as variations in electrode location. The long-term goal is a system that integrates high-level commands from the user with signals from implanted sensors to produce intelligent and adaptive motor function. Feasibility of this concept will be tested here for FES control of six muscles in the upper extremity, to perform the task of reaching in the horizontal plane. The following specific aims are proposed: (1) Implementation of RL control on a virtual arm with computer-generated commands and rewards, (2) RL control on a virtual arm, with commands and rewards given by a human operator, and (3) RL control of muscles in a paralyzed arm in two subjects with high cervical spinal cord injury, with commands and rewards given by the user via a head tracker based input device.