HCC-Small: RSVP IconCHAT - A Brain Computer Interface for Icon-based Communication

在这个项目中，PI将解决通过新颖和直观的计算机界面提高沟通速度，从而使严重运动和言语障碍(SMSI)患者能够通过书面和口头语言进行社交的挑战。可用于SMSI人群的增强通信技术通常产生的速度约为每分钟一个字(基于临床经验)。PI的目标是开发一种基于EEG的大脑接口技术，该技术基于基于图标的直观语言生成框架RSVP icCHAT，将实现目标人群更高的通信速率。这项技术将展示三个基本特征：代表单词的图标的快速连续可视呈现(RSVP)；能够准确预测预期文本的大词汇量自然语言模型，以便控制即将向受试者显示的图标序列，以便在RSVP范式中进行确认；以及融合来自多通道脑电(EEG)和生成概率语言模型的信息的意图检测机制。先进的统计信号处理、机器学习和自然语言建模技术将被用来实现比当前最先进水平高出一个数量级的通信速率。该项目还将为同步脑接口设计，特别是单次试验ERP检测贡献新的技术和算法。大脑接口和语言模型组件都将从之前与用户的交互中学习，并表现出稳健的合作学习行为，以最大限度地提高语言吞吐量。贝叶斯和信息论基础将支持适应性。PI指出，他的方法在三个维度上都是创新的：将使用基于图标的直观语言表示与上下文相关的语言模型相结合来构建消息；将开发用户自适应的非侵入性脑计算机界面，并将其用于与基于图标的平台的接口；将开发由BCI测量的大脑活动和预测语言模型之间的概率信息融合方法。广泛影响：由于各种原因，如脑瘫(CP)、神经肌肉疾病(肌萎缩侧索硬化症，ALS)和导致闭锁综合征(LIS)的严重脊髓损伤，存在大量SMSI人群。这些社区依赖于低效的通信模式，限制了用户产生可接受的通信速率的能力。该项目目标的成功实现不仅将为目标人群提供与其健全的沟通伙伴进行面对面交流的更好体验，而且还将使他们能够控制环境和获取信息。此外，这项工作将有助于不同模式的信息融合、最佳数据降维、单次试验ERP检测以及通过新的界面进行人机交流。在实验中收集的数据将提供给其他研究人员，以加快结果的核实和结果的传播。

In this project the PI will address the challenge of empowering people with severe motor and speech impairments (SMSI) to socialize through written and spoken language, by increasing communication rate through a novel and intuitive computer interface. Available augmented communication technologies for the SMSI population typically yield speeds on the order of just one word per minute (based on clinical experience). The PI's objective is to develop an EEG-based brain interface technology based on an intuitive icon-based language generation framework, RSVP iconCHAT, which will achieve increased communication rates for the target population. This technology will exhibit three essential features: rapid serial visual presentation (RSVP) of icons that represent words; a large-vocabulary natural language model with the capability for accurate predictions of intended text in order to control the upcoming sequence of icons to be shown to the subject for confirmation in the RSVP paradigm; and an intent detection mechanism that fuses information from multichannel electroencephalography (EEG) and the generative probabilistic language model. Advanced statistical signal processing, machine learning, and natural language modeling techniques will be employed to achieve communication rates over an order of magnitude higher than the current state-of-the-art. The project will also contribute novel techniques and algorithms for synchronous brain interface design, particularly single-trial ERP detection. Both the brain interface and language model components will learn from previous interactions with the user and exhibit robust cooperative learning behavior in order to maximize language throughput. A Bayesian and information theoretic foundation will support adaptability. The PI notes that his approach is innovative along three dimensions: an intuitive icon-based language representation combined with context-dependent language models will be employed for message construction; a noninvasive brain computer interface that is user-adaptive will be developed and employed to interface with the icon-based platform; and methods for probabilistic information fusion between the brain activity measured by the BCI and the predictive language model will be developed.Broader Impacts: There exists a significant SMSI population due to various reasons such as cerebral palsy (CP), neuromuscular disease (Amyotrophic Lateral Sclerosis, ALS), and severe spinal cord injury leading to locked-in syndrome (LIS). These communities rely on inefficient modes of communication that limit the user's ability to generate acceptable communication rates. Successful achievement of this project's goals will not only provide the target population with an improved face-to-face communication experience with their able-bodied communication partners, but will also enable control of their environment and access to information. In addition, the work will contribute to information fusion from different modalities, optimal data dimensionality reduction, single-trial ERP detection, and human computer communication through a novel interface. Data collected in experiments will be made available to other researchers in order to accelerate verification of outcomes and dissemination of results.