HCC-Small: RSVP IconCHAT - A Brain Computer Interface for Icon-based Communication

在这个项目中，PI将通过一种新颖直观的计算机界面来提高交流率，从而使严重运动和语言障碍（SMSI）患者能够通过书面和口头语言进行社交。SMSI人群可用的增强通信技术通常产生的速度仅为每分钟一个单词（基于临床经验）。PI的目标是开发一种基于直观的基于图标的语言生成框架（RSVP iconCHAT）的基于脑电图的脑接口技术，这将为目标人群实现更高的通信速率。这项技术将展示三个基本特征：代表单词的图标的快速串行视觉呈现（RSVP）；一个大词汇量的自然语言模型，具有准确预测预期文本的能力，以便在RSVP范式中控制即将显示给受试者确认的图标序列；以及融合多通道脑电图（EEG）和生成概率语言模型信息的意图检测机制。先进的统计信号处理、机器学习和自然语言建模技术将被用来实现比目前最先进的技术高出一个数量级的通信速率。该项目还将为同步脑接口设计提供新技术和算法，特别是单次试验ERP检测。大脑接口和语言模型组件都将从以前与用户的交互中学习，并表现出强大的合作学习行为，以最大限度地提高语言吞吐量。贝叶斯和信息论的基础将支持适应性。PI指出，他的方法在三个方面是创新的：一个直观的基于图标的语言表示与上下文相关的语言模型相结合，将用于信息构建；将开发一种用户自适应的非侵入性脑机接口，并将其用于与基于图标的平台进行接口；以及脑机接口测量的脑活动与预测语言模型之间的概率信息融合方法。更广泛的影响：由于脑瘫（CP）、神经肌肉疾病（肌萎缩侧索硬化症（ALS））和严重脊髓损伤导致闭锁综合征（LIS）等各种原因，存在大量的SMSI人群。这些社区依赖于低效的通信模式，限制了用户产生可接受的通信速率的能力。该项目的目标的成功实现不仅将为目标人群提供更好的与健全的交流伙伴面对面交流的体验，而且还将使他们能够控制环境和获取信息。此外，该工作将有助于不同模式的信息融合，最佳数据降维，单次试验ERP检测以及通过新界面的人机通信。在实验中收集的数据将提供给其他研究人员，以便加速结果的核查和结果的传播。

In this project the PI will address the challenge of empowering people with severe motor and speech impairments (SMSI) to socialize through written and spoken language, by increasing communication rate through a novel and intuitive computer interface. Available augmented communication technologies for the SMSI population typically yield speeds on the order of just one word per minute (based on clinical experience). The PI's objective is to develop an EEG-based brain interface technology based on an intuitive icon-based language generation framework, RSVP iconCHAT, which will achieve increased communication rates for the target population. This technology will exhibit three essential features: rapid serial visual presentation (RSVP) of icons that represent words; a large-vocabulary natural language model with the capability for accurate predictions of intended text in order to control the upcoming sequence of icons to be shown to the subject for confirmation in the RSVP paradigm; and an intent detection mechanism that fuses information from multichannel electroencephalography (EEG) and the generative probabilistic language model. Advanced statistical signal processing, machine learning, and natural language modeling techniques will be employed to achieve communication rates over an order of magnitude higher than the current state-of-the-art. The project will also contribute novel techniques and algorithms for synchronous brain interface design, particularly single-trial ERP detection. Both the brain interface and language model components will learn from previous interactions with the user and exhibit robust cooperative learning behavior in order to maximize language throughput. A Bayesian and information theoretic foundation will support adaptability. The PI notes that his approach is innovative along three dimensions: an intuitive icon-based language representation combined with context-dependent language models will be employed for message construction; a noninvasive brain computer interface that is user-adaptive will be developed and employed to interface with the icon-based platform; and methods for probabilistic information fusion between the brain activity measured by the BCI and the predictive language model will be developed.Broader Impacts: There exists a significant SMSI population due to various reasons such as cerebral palsy (CP), neuromuscular disease (Amyotrophic Lateral Sclerosis, ALS), and severe spinal cord injury leading to locked-in syndrome (LIS). These communities rely on inefficient modes of communication that limit the user's ability to generate acceptable communication rates. Successful achievement of this project's goals will not only provide the target population with an improved face-to-face communication experience with their able-bodied communication partners, but will also enable control of their environment and access to information. In addition, the work will contribute to information fusion from different modalities, optimal data dimensionality reduction, single-trial ERP detection, and human computer communication through a novel interface. Data collected in experiments will be made available to other researchers in order to accelerate verification of outcomes and dissemination of results.