Investigating output modality for a brain-computer interface for communication

项目摘要 脑机接口（BCI）越来越多地用于科学研究和治疗。 严重残疾患者的干预应用。这种装置的典型使用提供了连续的 使用从头皮记录的大脑活动控制计算机光标和/或选择离散项 在脑表面（皮层电描记术）或脑内（皮层内 微电极记录）。作为治疗设备，脑机接口在恢复通信方面非常成功 瘫痪和静音患者的能力，最常见的是离散选择字母拼写。在一项研究中， 语音合成器的控制用于通过BCI的通信。然而，这项研究依赖于 侵入性神经学记录，这对于广泛的言语治疗应用是不切实际的。的 拟议的项目通过调查BCI的性能来解决这一知识差距，该BCI旨在映射非 侵入性EEG感觉运动节律转化为低维语音表示，特别是共振峰 频率，用于控制具有瞬时听觉和视觉反馈的连续元音合成器。一 一个原型装置已经被开发出来，并被一个飞行员用来控制一个二维共振峰 频率元音合成器该器械将用于人体受试者研究，以评价BCI 在元音产生任务中的表现主要解决两个研究课题：[1]反馈的影响 模式（听觉与视觉）对学习BCI控制和[2]调查长期保留的性能 使用脑机接口设备本研究的目的是说明有意义的听觉的可行性 反馈（即元音声音）作为用于语音BCI的适当反馈机制。这项技术将 最终使严重沟通障碍的患者受益，特别是对于那些侵入性BCI 不是可行的选择。这项研究的结果也将为未来的BCI设计提供基础， 使用非侵入性连续产生辅音和元音的三维语音合成器 脑电图长期的研究目标，提供连续的人工语音（包括元音和辅音） 实时合成用于恢复通信，将是我们提高 提高患者的生活质量，让他们更好地参与社会交往。

PROJECT SUMMARY Brain-computer interfaces (BCIs) are increasingly being used in both scientific research and therapeutic intervention applications for severely disabled patients. Typical usage of such devices provides continuous control over computer cursors and/or selection of discrete items using brain activity recorded from the scalp (electroencephalography), on the brain surface (electrocorticography), or inside the brain (intracortical microelectrode recording). As therapeutic devices, BCIs have been very successful in restoring communication abilities to paralyzed and mute patients, most often by discrete-choice letter spelling. In one study, real-time control of a speech synthesizer was employed for communication by BCI. However, this study relied on invasive neurological recordings, which is impractical for widespread speech therapeutic application. The proposed project addresses this knowledge gap by investigating performance of a BCI designed to map non- invasive EEG sensorimotor rhythms into a low-dimensional speech representation, specifically formant frequencies, for control of a continuous vowel synthesizer with instantaneous auditory and visual feedback. A prototype device has been developed and used by a pilot subject to control a two dimensional formant frequency vowel synthesizer. This device will be used in a human subjects study for evaluation of BCI performance in vowel production tasks to address two major research topics: [1] the effect of feedback modality (auditory vs. visual) on learning BCI control and [2] investigating long-term retention of performance using the BCI device. The goal of this research study is to illustrate the feasibility of meaningful auditory feedback (i.e. vowel sounds) as an appropriate feedback mechanism for a speech BCI. This technology will ultimately benefit patients with severe communication impairment, especially for those whom invasive BCIs are not viable options. The results of this research will also provide a basis for future BCI designs using low- dimensional speech synthesizers for continuous production of both consonants and vowels using non-invasive EEG. The long-term research goal, of providing continuous artificial speech (both vowels and consonants) synthesis in real-time for restoration of communication, will be a significant advance in our ability to improve the patients' quality of life and allow them to better engage in social interactions.