Towards Closed-Loop Speech Synthesis from Stereotactic EEG: A Unit Selection Approach
Towards Closed-Loop Speech Synthesis from Stereotactic EEG: A Unit Selection Approach
复制标题
从立体定向脑电图实现闭环语音合成：一种单元选择方法
DOI：
10.1109/icassp43922.2022.9747300
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Kubben, Pieter L.
中科院分区：
文献类型：
--
作者：
Angrick, Miguel;Ottenhoff, Maarten;Diener, Lorenz;Ivucic, Darius;Ivucic, Gabriel;Goulis, Sophocles;Colon, Albert J.;Wagner, Louis;Krusienski, Dean J.;Kubben, Pieter L.
神经系统疾病会严重影响语言交流。最近，已经提出了神经语音假体，其从皮层表面记录的神经信号重建可理解的语音。到目前为止，还不清楚类似的重建是否是可行的，从更深的大脑结构，以及是否可以直接从这些重建与低延迟合成可听语音，所需的一个实用的语音神经假体。本研究旨在应对这两项挑战。首先，我们实现了一个基于低延迟单元选择的合成器，它将神经信号转换为可听语音。第二，我们评估我们的方法从5例植入立体定向深度电极谁进行了荷兰语发音朗读任务的开环录音。我们实现的相关系数显着高于机会水平高达0.6和6.6毫秒的平均计算成本为每10毫秒帧。虽然目前重建的话语是不可理解的，我们的研究结果表明有前途的解码和运行时的能力，适合在闭环实验中的语音过程的调查。
Neurological disorders can severely impact speech communication. Recently, neural speech prostheses have been proposed that reconstruct intelligible speech from neural signals recorded superficially on the cortex. Thus far, it has been unclear whether similar reconstruction is feasible from deeper brain structures, and whether audible speech can be directly synthesized from these reconstructions with low-latency, as required for a practical speech neuroprosthetic. The present study aims to address both challenges. First, we implement a low-latency unit selection based synthesizer that converts neural signals into audible speech. Second, we evaluate our approach on open-loop recordings from 5 patients implanted with stereotactic depth electrodes who conducted a read-aloud task of Dutch utterances. We achieve correlation coefficients significantly higher than chance level of up to 0.6 and an average computational cost of 6.6 ms for each 10 ms frames. While the current reconstructed utterances are not intelligible, our results indicate promising decoding and run-time capabilities that are suitable for investigations of speech processes in closed-loop experiments.