CPS: Synergy: Collaborative Research: Closed-loop Hybrid Exoskeleton utilizing Wearable Ultrasound Imaging Sensors for Measuring Fatigue

CPS：协同：协作研究：利用可穿戴超声成像传感器测量疲劳的闭环混合外骨骼

• 批准号: 1646009
• 负责人: Kang Kim
• 金额: $ 40万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1646009&HistoricalAwards=false
• 关键词: CPS; Synergy; Collaborative; Research; Closed

The goal of this project is to develop an automated assistive device capable of restoring walking and standing functions in persons with motor impairments. Although research on assistive devices, such as active and passive orthoses and exoskeletons, has been ongoing for several decades, the improvements in mobility have been modest due to a number of limitations. One major challenge has been the limited ability to sense and interpret the state of the human, including volitional motor intent and fatigue. The proposed device will consist of powered electric motors, as well as the power generated by the person's own muscles. This work proposes to develop novel sensors to monitor muscle function, and, muscle fatigue is identified, the system will switch to the electric motors until the muscles recover. Through research on methods of seamless automated control of a hybrid assistive device while minimizing muscle fatigue, this study addresses significant limitations of prior work. The proposed project has the long-term potential to significantly improve walking and quality of life of individuals with spinal cord injuries and stroke. The proposed work will also contribute to new science of cyber-physical systems by integrating wearable image-based biosensing with physical exoskeleton systems through computational algorithms. This project will provide immersive interdisciplinary training for graduate and undergraduate students to integrate computational methods with imaging, robotics, human functional activity and artificial devices for solving challenging public health problems. A strong emphasis will be placed on involving undergraduate students in research as part of structured programs at our institutions. Additionally, students with disabilities will be involved in this research activities by leveraging an ongoing NSF-funded project. This project includes the development of wearable ultrasound imaging sensors and real-time image analysis algorithms that can provide direct measurement of the function and status of the underlying muscles. This will allow development of dynamic control allocation algorithms that utilize this information to distribute control between actuation and stimulation. This approach for closed-loop control based on muscle-specific feedback represents a paradigm shift from conventional lower extremity exoskeletons that rely only on joint kinematics for feedback. As a testbed for this new approach, the team will utilize a hybrid exoskeleton that combines active joint actuators with functional electrical stimulation of a person's own muscles. Repetitive electrical stimulation leads to the rapid onset of muscle fatigue that limits the utility of these hybrid systems and potentially increases risk of injury. The goals of the project are: develop novel ultrasound sensing technology and image analysis algorithms for real-time sensing of muscle function and fatigue; investigate closed-loop control allocation algorithms utilizing measured muscle contraction rates to minimize fatigue; integrate sensing and control methods into a closed loop hybrid exoskeleton system and evaluate on patients with spinal cord injury. The proposed approach will lead to innovative CPS science by (1) integrating a human-in-the-loop physical exoskeleton system with novel image-based real-time robust sensing of complex time-varying physical phenomena, such as dynamic neuromuscular activity and fatigue, and (2) developing novel computational models to interpret such phenomena and effectively adapt control strategies. This research will enable practical wearable image-based biosensing, with broader applications in healthcare. This framework can be widely applicable in a number of medical CPS problems that involve a human in the loop, including upper and lower extremity prostheses and exoskeletons, rehabilitation and surgical robots. The new control allocation algorithms relying on sensor measurements could have broader applicability in fault-tolerant and redundant actuator systems, and reliable fault-tolerant control of unmanned aerial vehicles.

该项目的目标是开发一种能够恢复运动障碍者行走和站立功能的自动辅助装置。虽然对诸如主动和被动矫形器和外骨骼等辅助装置的研究已经进行了几十年，但由于一些限制，移动性的改善一直是适度的。一个主要的挑战一直是有限的能力，感觉和解释的状态，包括意志运动的意图和疲劳。拟议中的设备将包括电动马达，以及由人自己的肌肉产生的电力。这项工作提出开发新型传感器来监测肌肉功能，并且，肌肉疲劳被识别，系统将切换到电动机，直到肌肉恢复。通过对混合辅助设备的无缝自动控制方法的研究，同时最大限度地减少肌肉疲劳，这项研究解决了以前工作的重大局限性。拟议的项目具有长期潜力，可以显着改善脊髓损伤和中风患者的行走和生活质量。拟议的工作还将通过计算算法将可穿戴的基于图像的生物传感与物理外骨骼系统相结合，为网络物理系统的新科学做出贡献。该项目将为研究生和本科生提供沉浸式跨学科培训，将计算方法与成像，机器人技术，人类功能活动和人工设备相结合，以解决具有挑战性的公共卫生问题。重点将放在让本科生参与研究，作为我们机构结构化课程的一部分。此外，残疾学生将通过利用正在进行的NSF资助的项目参与这项研究活动。该项目包括开发可穿戴超声成像传感器和实时图像分析算法，可以直接测量底层肌肉的功能和状态。这将允许开发动态控制分配算法，该算法利用该信息在致动和刺激之间分配控制。这种基于肌肉特定反馈的闭环控制方法代表了仅依赖于关节运动学反馈的传统下肢外骨骼的范式转变。作为这种新方法的试验平台，该团队将利用一种混合外骨骼，将主动关节致动器与人体自身肌肉的功能性电刺激相结合。重复电刺激导致肌肉疲劳的快速发作，这限制了这些混合系统的实用性，并可能增加受伤的风险。该项目的目标是：开发新的超声传感技术和图像分析算法，用于实时感知肌肉功能和疲劳;利用测量的肌肉收缩率研究闭环控制分配算法，以最大限度地减少疲劳;将传感和控制方法集成到闭环混合外骨骼系统中，并对脊髓损伤患者进行评估。所提出的方法将导致创新的CPS科学（1）将人在回路物理外骨骼系统与复杂时变物理现象（如动态神经肌肉活动和疲劳）的新型基于图像的实时鲁棒感测相结合，以及（2）开发新型计算模型来解释这些现象并有效地适应控制策略。这项研究将实现实用的可穿戴图像生物传感，在医疗保健领域具有更广泛的应用。该框架可以广泛应用于许多涉及人类的医疗CPS问题，包括上肢和下肢假肢和外骨骼，康复和手术机器人。新的控制分配算法依赖于传感器的测量，可以有更广泛的适用性，容错和冗余执行器系统，可靠的容错控制的无人机。

项目成果

Switched control of an N-degree-of-freedom input delayed wearable robotic system

• DOI: 10.1016/j.automatica.2020.109455
• 发表时间: 2021-01-16
• 期刊: AUTOMATICA
• 影响因子: 6.4
• 作者: Sheng, Zhiyu;Sun, Ziyue;Sharma, Nitin
• 通讯作者: Sharma, Nitin

Muscle Fatigue Assessment in a Wearable Neuroprosthesis Using Ultrasound Strain Imaging

使用超声应变成像评估可穿戴神经假体的肌肉疲劳

• 发表时间: 2019
• 期刊: Internationl Functional Electrical Stimulation Soceity Conference
• 作者: Zhiyu Sheng, Nitin Sharma

Hybrid Dynamical System Model and Robust Control of a Hybrid Neuroprosthesis Under Fatigue Based Switching

基于疲劳切换的混合动力系统模型和混合神经假体的鲁棒控制

• DOI: 10.23919/acc.2018.8431258
• 发表时间: 2018
• 期刊: American Control Conference
• 作者: Sheng, Zhiyu;Molazadeh, Vahidreza;Sharma, Nitin
• 通讯作者: Sharma, Nitin

NEURAL-NETWORK BASED ITERATIVE LEARNING CONTROL OF A HYBRID EXOSKELETON WITH AN MPC ALLOCATION STRATEGY

基于神经网络的具有 MPC 分配策略的混合外骨骼迭代学习控制

• 发表时间: 2019
• 期刊: Proceedings of the ASME 2019 Dynamic Systems and Control Conference
• 作者: Vahidreza Molazadeh, Qiang Zhang

Quantitative Assessment of Changes in Muscle Contractility Due to Fatigue During NMES: An Ultrasound Imaging Approach

• DOI: 10.1109/tbme.2019.2921754
• 发表时间: 2020-03-01
• 期刊: IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
• 影响因子: 4.6
• 作者: Sheng，Zhiyu;Sharma，Nitin;Kim，Kang
• 通讯作者: Kim，Kang