CPS: Synergy: A Novel Biomechatronic Interface Based on Wearable Dynamic Imaging Sensors

CPS：Synergy：基于可穿戴动态成像传感器的新型生物机电接口

• 批准号: 1329829
• 负责人: Siddhartha Sikdar
• 金额: $ 99.51万
• 依托单位: George Mason University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-02-01 至 2019-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1329829&HistoricalAwards=false
• 关键词: CPS; Synergy; Novel; Biomechatronic; Interface

The problem of controlling biomechatronic systems, such as multiarticulating prosthetic hands, involves unique challenges in the science and engineering of Cyber Physical Systems (CPS), requiring integration between computational systems for recognizing human functional activity and intent and controlling prosthetic devices to interact with the physical world. Research on this problem has been limited by the difficulties in noninvasively acquiring robust biosignals that allow intuitive and reliable control of multiple degrees of freedom (DoF). The objective of this research is to investigate a new sensing paradigm based on ultrasonic imaging of dynamic muscle activity. The synergistic research plan will integrate novel imaging technologies, new computational methods for activity recognition and learning, and high-performance embedded computing to enable robust and intuitive control of dexterous prosthetic hands with multiple DoF. The interdisciplinary research team involves collaboration between biomedical engineers, electrical engineers and computer scientists. The specific aims are to: (1) research and develop spatio-temporal image analysis and pattern recognition algorithms to learn and predict different dexterous tasks based on sonographic patterns of muscle activity (2) develop a wearable image-based biosignal sensing system by integrating multiple ultrasound imaging sensors with a low-power heterogeneous multicore embedded processor and (3) perform experiments to evaluate the real-time control of a prosthetic hand.The proposed research methods are broadly applicable to assistive technologies where physical systems, computational frameworks and low-power embedded computing serve to augment human activities or to replace lost functionality. The research will advance CPS science and engineering through integration of portable sensors for image-based sensing of complex adaptive physical phenomena such as dynamic neuromuscular activity, and real-time sophisticated image understanding algorithms to interpret such phenomena running on low-power high performance embedded systems. The technological advances would enable practical wearable image-based biosensing, with applications in healthcare, and the computational methods would be broadly applicable to problems involving activity recognition from spatiotemporal image data, such as surveillance.This research will have societal impacts as well as train students in interdisciplinary methods relevant to CPS. About 1.6 million Americans live with amputations that significantly affect activities of daily living. The proposed project has the long-term potential to significantly improve functionality of upper extremity prostheses, improve quality of life of amputees, and increase the acceptance of prosthetic limbs. This research could also facilitate intelligent assistive devices for more targeted neurorehabilitation of stroke victims. This project will provide immersive interdisciplinary CPS-relevant training for graduate and undergraduate students to integrate computational methods with imaging, processor architectures, human functional activity and artificial devices for solving challenging public health problems. A strong emphasis will be placed on involving undergraduate students in research as part of structured programs at our institution. The research team will involve students with disabilities in research activities by leveraging an ongoing NSF-funded project. Bioengineering training activities will be part of a newly developed undergraduate curriculum and a graduate curriculum under development.The synergistic research plan has been designed to advance CPS science and engineering through the development of new computational methods for dynamic activity recognition and learning from image sequences, development of novel wearable imaging technologies including high-performance embedded computing, and real-time control of a physical system. The specific aims are to: (1) Research and develop spatio-temporal image analysis and pattern recognition algorithms to learn and predict different dexterous tasks based on sonographic patterns of muscle activity. The first aim has three subtasks designed to collect, analyze and understand image sequences associated with functional tasks. (2) Develop a wearable image-based biosignal sensing system by integrating multiple ultrasound imaging sensors with a low-power heterogeneous multicore embedded processor. The second aim has two subtasks designed to integrate wearable imaging sensors with a real-time computational platform. (3) Perform experiments to evaluate the real-time control of a prosthetic hand. The third aim will integrate the wearable image acquisition system developed in Aim 2, and the image understanding algorithms developed in Aim 1, for real-time evaluation of the control of a prosthetic hand interacting with a virtual reality environment.Successful completion of these aims will result in a real-time system that acquires image data from complex neuromuscular activity, decodes activity intent from spatiotemporal image data using computational algorithms, and controls a prosthetic limb in a virtual reality environment in real time. Once developed and validated, this system can be the starting point for developing a new class of sophisticated control algorithms for intuitive control of advanced prosthetic limbs, new assistive technologies for neurorehabilitation, and wearable real-time imaging systems for smart health applications.

控制生物机电系统的问题，如多关节假肢手，涉及网络物理系统（CPS）科学和工程中的独特挑战，需要在计算系统之间进行集成，以识别人类功能活动和意图，并控制假肢设备与物理世界进行交互。对这一问题的研究受到无创获取鲁棒生物信号的困难的限制，这些信号可以直观可靠地控制多个自由度（DoF）。本研究的目的是探讨一种基于动态肌肉活动超声成像的新传感范式。协同研究计划将集成新的成像技术、用于活动识别和学习的新计算方法以及高性能嵌入式计算，以实现具有多个自由度的灵巧假肢手的鲁棒和直观控制。这个跨学科的研究团队包括生物医学工程师、电气工程师和计算机科学家之间的合作。具体目标是：(1)研究和开发基于肌肉活动超声模式的时空图像分析和模式识别算法，以学习和预测不同的灵巧任务；(2)通过集成多个超声成像传感器和低功耗异构多核嵌入式处理器，开发基于图像的可穿戴生物信号传感系统；(3)进行实验以评估假手的实时控制。所提出的研究方法广泛适用于辅助技术，其中物理系统，计算框架和低功耗嵌入式计算用于增强人类活动或取代失去的功能。该研究将通过集成便携式传感器，用于基于图像的复杂自适应物理现象（如动态神经肌肉活动）的传感，以及在低功耗高性能嵌入式系统上运行的实时复杂图像理解算法来解释这些现象，从而推进CPS科学和工程。技术进步将使实际的可穿戴图像生物传感应用于医疗保健，计算方法将广泛适用于涉及从时空图像数据识别活动的问题，例如监测。这项研究将产生社会影响，并训练学生使用与CPS相关的跨学科方法。大约有160万美国人截肢，严重影响了日常生活活动。本项目具有显著改善上肢义肢功能、改善截肢者生活质量、提高义肢接受度的长期潜力。这项研究还可以促进智能辅助设备，为中风患者提供更有针对性的神经康复。该项目将为研究生和本科生提供沉浸式跨学科cps相关培训，将计算方法与成像、处理器架构、人体功能活动和人工设备相结合，以解决具有挑战性的公共卫生问题。重点将放在让本科生参与研究，作为我们机构结构化课程的一部分。研究小组将利用正在进行的nsf资助项目，让残疾学生参与研究活动。生物工程培训活动将成为新开发的本科课程和正在开发的研究生课程的一部分。协同研究计划旨在通过开发用于动态活动识别和从图像序列中学习的新计算方法、开发新型可穿戴成像技术（包括高性能嵌入式计算）和物理系统的实时控制来推进CPS科学和工程。具体目标是：(1)研究和开发基于肌肉活动超声模式的时空图像分析和模式识别算法，以学习和预测不同的灵巧任务。第一个目标有三个子任务，旨在收集、分析和理解与功能任务相关的图像序列。(2)将多个超声成像传感器与低功耗异构多核嵌入式处理器集成，开发基于图像的可穿戴生物信号传感系统。第二个目标有两个子任务，旨在将可穿戴成像传感器与实时计算平台集成在一起。(3)进行实验，评估假手的实时控制。第三个目标将集成aim 2开发的可穿戴图像采集系统和aim 1开发的图像理解算法，用于实时评估假手与虚拟现实环境交互的控制。这些目标的成功完成将产生一个实时系统，该系统从复杂的神经肌肉活动中获取图像数据，使用计算算法从时空图像数据中解码活动意图，并在虚拟现实环境中实时控制假肢。一旦开发和验证，该系统可以成为开发新型复杂控制算法的起点，用于直观控制先进假肢，神经康复的新型辅助技术，以及用于智能健康应用的可穿戴实时成像系统。

项目成果

An intuitive muscle-computer interface using ultrasound sensing and Markovian state transitions

使用超声波传感和马尔可夫状态转换的直观肌肉计算机界面

• DOI: 10.1109/isbi.2018.8363784
• 发表时间: 2018
• 期刊: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018
• 作者: Dhawan, Ananya S.;Kosecka, Jana;Rangwala, Huzefa;Sikdar, Siddhartha
• 通讯作者: Sikdar, Siddhartha