A Hybrid Brain-Computer Interface for Long-Term Use by Persons with Severe Motor Deficit: Towards Development of Personalized Algorithms

供严重运动缺陷患者长期使用的混合脑机接口：面向个性化算法的开发

• 批准号: 1913492
• 负责人: Yalda Shahriari
• 金额: $ 24.96万
• 依托单位: University of Rhode Island
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1913492&HistoricalAwards=false
• 关键词: Hybrid; Brain; Computer; Interface; Long

The inability to communicate is a major consequence of severe motor disabilities such as amyotrophic lateral sclerosis (ALS), also known as Lou Gehrig's disease. ALS is the most common adult-onset motor neuron disease. Within the past three decades, Brain-Computer Interface (BCI) technologies have emerged as an alternative communication channel for persons with neuromuscular disorders. However, these communication systems typically require adequate eye-gaze control, which is difficult for those lacking voluntary muscle control. In addition, most current BCIs are inadequate for long-term use due to significant day-to-day variation in performance. To address these shortcomings, this project will combine two technologies to monitor the electrical properties of the neurons and the blood flow near the neurons. This combined system will adapt as the disease progresses and improve communication for persons with ALS. If successful, the system may be generalized to other types of motor control loss, including minimally conscious and vegetative states. The technologies developed will be integrated into a university curriculum that educates undergraduate and graduate students about the field. Furthermore, this research will be broadly disseminated through k-12 curriculum design and will provide training opportunities for women and under-represented minorities. Outreach activities include workshops: "Personalized Algorithms in BCI Systems" and "Hybrid BCI for Bedside and Home Users," poster presentations to disseminate novel outcomes from this study among students and researchers and development of a K-12 curriculum, "Engineering the Brain," designed to promote BCI learning opportunities among middle school students.The project focuses on designing Brain Computer Interfaces (BCIs) based on functional imaging of the human brain using a combination of two non-invasive techniques: electroencephalography (EEG-to measure electrical activity) and functional near-infrared spectroscopy (fNIRS-to measure hemodynamic activity). Algorithms will be developed such that the system considers both the neuropsychological status and environmental factors and can be implemented for in-home, long-term care of people with no motor control. The system will be tested on persons with ALS (amyotrophic lateral sclerosis), the most common adult-onset motor neuron disease. The project's hypothesis is that the incorporation of hemodynamic activities into conventional EEG-based BCI permits superior learning from brain states, provides unique features of the user's intent, and allows potentially game-changing solutions to provide end-users with a new level of communication autonomy. To test this hypothesis, the Research Plan is organized under two objectives. The FIRST Objective is to explore the associations of BCI performance variations and design a multimodal augmented predictive platform to improve the robustness of the BCI systems for nonverbal patients who have residual motor ability to control their eye-gaze. The two aims of the objective are to explore the associations of ALS-BCI performance variation with both functional brain changes in ALS and environmental noise and to establish a set of predictive electro-vascular features best representative of users' BCI performance, and accordingly correct for unrelated activities in future BCI experiments. Expected objective outcomes include (i) longitudinal assessment of brain pattern changes, (ii) establishing a set of predictive electro-vascular features, (iii) optimizing subject-specific factors essential for successful BCI performance, (iv) employing appropriate correction strategies to minimize unwanted activities and (v) maximizing adaptability with users' needs and environmental factors while minimizing inter-subject variabilities. The Second Objective is to develop an autonomous hybrid BCI for non-communicative persons without any residual motor control (preferably participants in the first objective who have progressed to the locked-in stage.) The system developed will have seven Degrees of Freedom (DOF) that can produce seven different commands for an external device based on the users' ability to modulate their brain signals through motor imagery (5 tasks), mental arithmetic (1 task) and rest. The expected objective outcome is the introduction of a hybrid BCI that (i) is fully autonomous-signal-based, (ii) employs personalized techniques to enhance system performance, (iii) has enhanced information transfer rates relative to single modal EEG and fNIRS and (iv) can be conveniently set up at users' bedsides for long-term use.This project is jointly funded by the Disabilities and Rehabilitation Engineering (DARE) program and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

无法交流是严重运动障碍的主要后果，如肌萎缩侧索硬化症（ALS），也被称为Lou Gehrig's病。肌萎缩侧索硬化症是最常见的成人发病的运动神经元疾病。在过去的三十年里，脑机接口（BCI）技术已经成为神经肌肉疾病患者的另一种交流渠道。然而，这些通信系统通常需要足够的眼睛凝视控制，这对于那些缺乏随意肌肉控制的人来说是困难的。此外，目前大多数脑机接口不适合长期使用，因为其性能每天都有显著变化。为了解决这些缺点，该项目将结合两种技术来监测神经元的电特性和神经元附近的血液流动。这种组合系统将随着疾病的进展而适应，并改善ALS患者的沟通。如果成功，该系统可以推广到其他类型的运动控制丧失，包括最低意识状态和植物人状态。所开发的技术将被整合到大学课程中，为本科生和研究生提供有关该领域的教育。此外，这项研究将通过k-12课程设计广泛传播，并将为妇女和代表性不足的少数民族提供培训机会。外展活动包括研讨会：“脑机接口系统中的个性化算法”和“床边和家庭用户的混合脑机接口”，在学生和研究人员中传播这项研究的新成果的海报展示，以及开发K-12课程“大脑工程”，旨在促进中学生的脑机接口学习机会。该项目侧重于设计基于人脑功能成像的脑机接口（bci），使用两种非侵入性技术的组合：脑电图（eeg -测量电活动）和功能近红外光谱（fnirs -测量血流动力学活动）。算法将被开发，使系统同时考虑神经心理状态和环境因素，并可以在家中实施，长期护理没有运动控制的人。该系统将在ALS（肌萎缩性侧索硬化症）患者身上进行测试，ALS是最常见的成人发病运动神经元疾病。该项目的假设是，将血流动力学活动整合到传统的基于脑电图的脑机接口中，可以从大脑状态中获得更好的学习，提供用户意图的独特特征，并允许潜在的改变游戏规则的解决方案，为最终用户提供一个新的沟通自主权水平。为了检验这一假设，研究计划是在两个目标下组织的。第一个目标是探索脑机接口性能变化的关联，并设计一个多模态增强预测平台，以提高脑机接口系统的鲁棒性，以帮助那些有剩余运动能力控制眼睛注视的非语言患者。本研究的两个目的是探讨ALS-BCI功能变化与ALS患者脑功能变化和环境噪声之间的关系，并建立一套最能代表用户BCI功能的预测性电血管特征，从而在未来的BCI实验中纠正不相关的活动。预期的客观结果包括(i)对脑模式变化的纵向评估，（ii）建立一套预测性电血管特征，（iii）优化脑机接口（BCI）成功表现所必需的受试者特定因素，（iv）采用适当的纠正策略以减少不必要的活动，(v)最大限度地适应用户需求和环境因素，同时最大限度地减少受试者之间的差异。第二个目标是为没有任何残余运动控制的非交流人员（最好是第一个目标中已经进展到锁定阶段的参与者）开发一个自主混合BCI。开发的系统将具有7个自由度（DOF），可以根据用户通过运动图像（5个任务）、心算（1个任务）和休息来调节大脑信号的能力，为外部设备产生7种不同的命令。预期的目标结果是引入混合脑机接口(i)完全基于自主信号，（ii）采用个性化技术来提高系统性能，（iii）相对于单模态EEG和fNIRS提高了信息传输速率，（iv）可以方便地设置在用户床边长期使用。该项目由残疾和康复工程（DARE）项目和促进竞争性研究的既定项目（EPSCoR）共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

A Graph-Based Nonlinear Dynamic Characterization of Motor Imagery Toward an Enhanced Hybrid BCI

• DOI: 10.1007/s12021-022-09595-2
• 发表时间: 2022-07
• 期刊: Neuroinformatics
• 影响因子: 3
• 作者: S. Hosni;S. B. Borgheai;J. McLinden;Shaotong Zhu;Xiaofei Huang;S. Ostadabbas;Y. Shahriari

A Graph-Based Feature Extraction Algorithm Towards a Robust Data Fusion Framework for Brain-Computer Interfaces

基于图的特征提取算法实现脑机接口的鲁棒数据融合框架

• 发表时间: 2021
• 期刊: 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society
• 作者: Zhu, Shaotong;Hosni, Sarah;McLinden, John;Borgheai, Bahram;Shahriari, Yalda;Ostadabbas, Sarah.
• 通讯作者: Ostadabbas, Sarah.

A Graph-Based Dynamical Characterization and Inference in Hybrid BCIs

混合 BCI 中基于图的动态表征和推理

• 发表时间: 2021
• 期刊: and Computers
• 作者: S. I Hosni, S. B.

Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework

• DOI: 10.1364/boe.413666
• 发表时间: 2021-03-01
• 期刊: BIOMEDICAL OPTICS EXPRESS
• 影响因子: 3.4
• 作者: Deligani, Roohollah Jafari;Borgheai, Seyyed Bahram;Shahriari, Yalda
• 通讯作者: Shahriari, Yalda

Improving longitudinal P300-BCI performance for people with ALS using a data augmentation and jitter correction approach

• DOI: 10.1080/2326263x.2021.2014678
• 发表时间: 2021-12
• 期刊: Brain-Computer Interfaces
• 影响因子: 2.1
• 作者: A. H. Zisk;S. B. Borgheai;J. McLinden;R. J. Deligani;Y. Shahriari