I-Corps: Dexterous Robotic Prosthetic Control Using Deep Learning Pattern Prediction from Ultrasound Signal

这个i-Corps项目的更广泛的影响/商业潜力在于开发了一种新的系统，该系统将允许接受经尺侧手部分截肢的人获得无与伦比的精确的个性化假肢手指运动，包括单个手指的连续和同时运动，而不需要漫长而复杂的训练过程。为了实现这种功能，设计了一组新的深度学习算法来模拟超声图像中的肌肉运动模式。网络预先训练了大量数据，以尽量减少后来的个体训练。除了动力假肢，拟议的技术还可以在其他需要机器人和/或数字环境的易于使用和准确的手势控制的市场上产生广泛的影响。其中包括远程机器人、外骨骼操作、虚拟现实、游戏、手套箱以及与工作相关的个人防护设备，以及性能增强和放大设备。这个i-Corps项目将开发和利用新型超声波传感器和新型深度学习算法来识别连续的肌肉活动模式，可以预测准确和灵活的手指运动。目前的肌电假体使用离散分类器，只能从噪声肌电(EMG)信号中预测有限数量的离散手势。为深度学习结构(如卷积神经网络)提供丰富和详细的超声信号，有望对详细的连续和同时肌肉运动模式进行建模和预测，这些模式可以映射到控制单个假肢手指的连续和同时运动。该项目的另一个优点是利用大量数据对这些深度神经网络进行预训练，这将允许对单个用户进行短期的微调培训，从而允许广泛和容易地采用该技术。因此，拟议的项目可以允许截肢者和上半身残障人士进行手指间的运动活动，例如精细操纵物体、打字或演奏乐器。

The broader impact/commercial potential of this I-Corps project lies in the development a novel system that would allow people with transradial and partial hand amputations to gain unparalleled precise individuation of prosthetic digit motion including continuous and simultaneous movement for individual digits without requiring long and complicated training process. To allow for such functionality a novel set of deep learning algorithms are designed to model muscle movement patterns from ultrasound images. The network is pre-trained with a large amount of data in an effort to minimize later individual training. In addition to power prosthetics, the proposed technology can provide broad impacts in other markets where easy-to-use and accurate gestural control of robotics and/or digital environment are required. These include tele-robotics, exoskeleton operation, virtual reality, gaming, glove boxes as well as work related Personal Protective Equipment, and Performance Augmentation and Amplification Devices.This I-Corps project will develop and utilize a novel ultrasound sensor and novel deep learning algorithms to recognize continuous muscle activity patterns that can predict accurate and dexterous finger motion. Current myoelectric powered prostheses use discrete classifiers that can only predict a limited number of discrete gestures from noisy electromyography (EMG) signal. Feeding deep learning architectures, such as Convolutional Neural Networks, with rich and detailed ultrasound signal promises to allow for the modeling and prediction of detailed continuous and simultaneous muscle movements patterns, which can be mapped to control continuous and simultaneous movements of individual prosthetic fingers. An additional intellectual of this project merit is the pre-training of these deep neural network with a large amount of data, which would allow for short fine tuning training for individual users, allowing for wide and easy adoption of the technology. The proposed project could therefore allow amputees and people with upper body disabilities to perform finger-by-finger movement activities such as fine object manipulation, typing or playing a musical instrument.