Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus.
Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus.
复制标题
DOI：
10.1088/1741-2552/abbfef
复制
发表时间：
2020-11-25
影响因子：
4
通讯作者：
Shenoy KV
中科院分区：
文献类型：
--
作者：
Wilson GH;Stavisky SD;Willett FR;Avansino DT;Kelemen JN;Hochberg LR;Henderson JM;Druckmann S;Shenoy KV
为了评估脑机接口 (BCI) 皮质内电极阵列信号恢复丢失语音的潜力，我们测量了经过训练以区分 39 个英语音素的综合基础集并通过神经模式匹配方法合成语音的解码器的性能。我们解码了中央前回“手旋钮”区域中大声说出的单词的神经相关性，这是朝着解码无法说话的患者腹侧言语区域尝试言语的最终目标迈出的一步。两名 BrainGate2 试点临床试验参与者（每人都有两个长期植入的 96 电极阵列）说出 420 个不同的单词，这些单词广泛采样了英语音素，同时记录了神经和音频数据。从录音中识别出音素起始点，然后根据由每个电极的分组动作电位计数或高频局部场电位功率组成的神经特征对它们的身份进行分类。使用“大脑到语音”模式匹配方法进行语音合成。我们还研究了解码公开语音时特有的两个潜在混淆：神经信号的声学污染以及标记不同音素出现时间的系统差异。线性解码器在 39 个音素上实现了高达 29.3% 的分类准确率（机会 = 6%），而循环神经网络分类器则实现了 33.9% 的准确率。参数扫描表明，当添加更多电极或更多训练数据时，性能不会饱和，并且当利用数据中的时变结构时，准确性会提高。麦克风污染和音素起始差异略微提高了解码准确性，但可以分别通过声学伪影扣除和使用神经语音起始标记来减轻。语音合成实现了真实音频和重构音频之间的 r = 0.523 相关性。使用来自非传统语音区域的皮层内电极阵列信号解码语音的能力表明，将电极阵列放置在腹侧语音区域是语音 BCI 的一个有前途的方向。
To evaluate the potential of intracortical electrode array signals for brain-computer interfaces (BCIs) to restore lost speech, we measured the performance of decoders trained to discriminate a comprehensive basis set of 39 English phonemes and to synthesize speech sounds via a neural pattern matching method. We decoded neural correlates of spoken-out-loud words in the “hand knob” area of precentral gyrus, a step towards the eventual goal of decoding attempted speech from ventral speech areas in patients who are unable to speak. Neural and audio data were recorded while two BrainGate2 pilot clinical trial participants, each with two chronically-implanted 96-electrode arrays, spoke 420 different words that broadly sampled English phonemes. Phoneme onsets were identified from audio recordings, and their identities were then classified from neural features consisting of each electrode’s binned action potential counts or high-frequency local field potential power. Speech synthesis was performed using the ‘Brain-to-Speech’ pattern matching method. We also examined two potential confounds specific to decoding overt speech: acoustic contamination of neural signals and systematic differences in labeling different phonemes’ onset times. A linear decoder achieved up to 29.3% classification accuracy (chance = 6%) across 39 phonemes, while a recurrent neural network classifier achieved 33.9% accuracy. Parameter sweeps indicated that performance did not saturate when adding more electrodes or more training data, and that accuracy improved when utilizing time-varying structure in the data. Microphonic contamination and phoneme onset differences modestly increased decoding accuracy, but could be mitigated by acoustic artifact subtraction and using a neural speech onset marker, respectively. Speech synthesis achieved r = 0.523 correlation between true and reconstructed audio. The ability to decode speech using intracortical electrode array signals from a nontraditional speech area suggests that placing electrode arrays in ventral speech areas is a promising direction for speech BCIs.
登录
查看更多内容
影响因子：
4
作者：
Chestek CA;Gilja V;Nuyujukian P;Foster JD;Fan JM;Kaufman MT;Churchland MM;Rivera-Alvidrez Z;Cunningham JP;Ryu SI;Shenoy KV
通讯作者：
Shenoy KV
细胞外田地和电流的起源-EEG，ECOG，LFP和尖峰。
DOI：
10.1038/nrn3241
发表时间：
2012-05-18
期刊：
Nature reviews. Neuroscience
影响因子：
--
作者：
Buzsáki G;Anastassiou CA;Koch C
通讯作者：
Koch C
影响因子：
7.7
作者：
Cheung, Connie;Hamiton, Liberty S.;Chang, Edward F.
通讯作者：
Chang, Edward F.
影响因子：
3.7
作者：
Chan, Alexander M.;Dykstra, Andrew R.;Cash, Sydney S.
通讯作者：
Cash, Sydney S.
影响因子：
--
作者：
Abadi, Martin
通讯作者：
Abadi, Martin