HCC-Small: RSVP IconCHAT - A Brain Computer Interface for Icon-based Communication

在这个项目中，PI将通过新颖直观的计算机界面提高沟通速度，解决使严重运动和语言障碍（SMSI）患者能够通过书面和口头语言进行社交的挑战。 适用于SMSI人群的现有增强通信技术通常产生大约每分钟一个字的速度（基于临床经验）。 PI的目标是开发一种基于EEG的大脑接口技术，该技术基于直观的基于图标的语言生成框架RSVP iconCHAT，这将提高目标人群的通信速率。 这项技术将展示三个基本特征：快速串行视觉呈现（RSVP）的图标，代表的话，一个大词汇量的自然语言模型，能够准确预测预期的文本，以控制即将出现的图标序列显示给主题确认在RSVP范例;以及融合来自多通道脑电图（EEG）和生成概率语言模型的信息的意图检测机制。 先进的统计信号处理，机器学习和自然语言建模技术将被用来实现通信速率超过一个数量级高于目前的最先进的。该项目还将贡献新的技术和算法同步脑接口设计，特别是单次试验ERP检测。 大脑接口和语言模型组件都将从先前与用户的交互中学习，并表现出强大的合作学习行为，以最大限度地提高语言吞吐量。 贝叶斯和信息理论基础将支持适应性。 PI指出，他的方法是创新的沿着沿三个维度：一个直观的图标为基础的语言表示结合上下文相关的语言模型将用于信息建设;一个非侵入性的脑机接口，是用户自适应的将开发和使用接口的图标为基础的平台;并将开发BCI测量的大脑活动与预测语言模型之间的概率信息融合方法。由于各种原因，如脑瘫（CP）、神经肌肉疾病（肌萎缩侧索硬化症，ALS）和导致闭锁综合征（LIS）的严重脊髓损伤，存在大量SMSI人群。 这些社区依赖于低效的通信模式，这限制了用户产生可接受的通信速率的能力。 成功实现这一项目的目标，不仅将使目标人群能够更好地与身体健全的交流伙伴进行面对面的交流，而且还将使他们能够控制自己的环境和获得信息。 此外，这项工作将有助于不同模态的信息融合，最佳数据降维，单次试验ERP检测，以及通过新颖的界面进行人机通信。 在实验中收集的数据将提供给其他研究人员，以加快结果的验证和结果的传播。

In this project the PI will address the challenge of empowering people with severe motor and speech impairments (SMSI) to socialize through written and spoken language, by increasing communication rate through a novel and intuitive computer interface. Available augmented communication technologies for the SMSI population typically yield speeds on the order of just one word per minute (based on clinical experience). The PI's objective is to develop an EEG-based brain interface technology based on an intuitive icon-based language generation framework, RSVP iconCHAT, which will achieve increased communication rates for the target population. This technology will exhibit three essential features: rapid serial visual presentation (RSVP) of icons that represent words; a large-vocabulary natural language model with the capability for accurate predictions of intended text in order to control the upcoming sequence of icons to be shown to the subject for confirmation in the RSVP paradigm; and an intent detection mechanism that fuses information from multichannel electroencephalography (EEG) and the generative probabilistic language model. Advanced statistical signal processing, machine learning, and natural language modeling techniques will be employed to achieve communication rates over an order of magnitude higher than the current state-of-the-art. The project will also contribute novel techniques and algorithms for synchronous brain interface design, particularly single-trial ERP detection. Both the brain interface and language model components will learn from previous interactions with the user and exhibit robust cooperative learning behavior in order to maximize language throughput. A Bayesian and information theoretic foundation will support adaptability. The PI notes that his approach is innovative along three dimensions: an intuitive icon-based language representation combined with context-dependent language models will be employed for message construction; a noninvasive brain computer interface that is user-adaptive will be developed and employed to interface with the icon-based platform; and methods for probabilistic information fusion between the brain activity measured by the BCI and the predictive language model will be developed.Broader Impacts: There exists a significant SMSI population due to various reasons such as cerebral palsy (CP), neuromuscular disease (Amyotrophic Lateral Sclerosis, ALS), and severe spinal cord injury leading to locked-in syndrome (LIS). These communities rely on inefficient modes of communication that limit the user's ability to generate acceptable communication rates. Successful achievement of this project's goals will not only provide the target population with an improved face-to-face communication experience with their able-bodied communication partners, but will also enable control of their environment and access to information. In addition, the work will contribute to information fusion from different modalities, optimal data dimensionality reduction, single-trial ERP detection, and human computer communication through a novel interface. Data collected in experiments will be made available to other researchers in order to accelerate verification of outcomes and dissemination of results.