Decoding Speech using Invasive Brain-Computer Interfaces based on Intracranial Brain Signals (dSPEECH)

一些患者不能说话，是因为在语言产生或运动中招募的神经通路受损或退化，如运动神经元疾病(MND)或肌萎缩侧索硬化症(ALS)。然而，如果负责语言或认知的大脑结构完好无损，脑机接口(BCI)可能会对他们有利，因为BCI有可能通过直接将大脑信号解码为语音来绕过受损的神经通路。BCI可以使用有创性或非侵入性的方式来记录大脑信号。一般情况下，脑电信号等非侵入式BCI所记录的脑信号质量较差，信噪比较低，因此目前非侵入式BCI无法以可接受的性能解码语音。与之不同的是，侵入性BCI如皮层脑电图仪(ECoG)和立体脑电图仪(SEEG)可以采集足够的时空分辨率的高质量的脑内信号，因此利用侵入性BCI进行语音解码是很有前途的。尽管近年来利用侵入性BCI(ECoG和SEEG)进行显性语音解码发展迅速，并产生了许多优秀的结果，但隐蔽(想象)语音解码仍然是一个具有挑战性的问题。这种情况背后有几个原因。第一个也是主要的原因是，与显性语音相比，关联的神经信号较弱且可变，因此机器学习算法很难解码隐蔽语音。第二个原因是无创性记录的有限的想象语音数据集。这种有限的数据集不能使用动物模型来缓解，因为动物使用的交流系统被认为仅限于表达有限数量的话语，这主要是由基因决定的。此外，对人体的录音通常仅限于非侵入性技术。颅内数据只能在临床环境中从耐药癫痫或其他神经相关疾病的患者那里获得。言语研究的纳入标准，如正常认知和发音能力，进一步减少了潜在受试者的数量。本项目(DSPEECH)旨在对上述因素进行一些突破，并进行一项新颖的研究。在dSPEECH中，我们雄心勃勃地希望使用侵入性BCI(ECoG和SEEG)来解码隐蔽语音。我们将建立新一代脑-文本BCI的新范式，开发用于解码隐蔽语音的先进机器学习/深度学习算法，并构建世界上第一个用于隐蔽语音的ECoG/SEEG数据集。此外，我们亦会处理可能出现的道德问题和资料管理问题。有了这样的ECoG/SEEG数据集和合适的解码方法，我们有信心在隐蔽语音解码的研究方面取得更大的进展。DSPEECH是一个联合项目，由多学科成员组成，包括来自神经工程的研究人员和来自临床医学的神经外科医生。我们还得到了包括著名BCI公司和海外经验丰富的神经外科医生在内的合作伙伴的大力支持。基于密切和扎实的合作，我们相信dSPEECH可以产生世界领先的结果。

Some patients cannot speak because of impairment or degeneration of neural pathways recruited in speech production or movement such as motor neurone disease (MND) or amyotrophic lateral sclerosis (ALS). However, brain-computer interfaces (BCIs) may benefit them if their brain structure responsible for language or cognition is intact, as BCIs have the potential to bypass damaged neural pathways by decoding brain signals into speech directly. BCIs may use invasive or non-invasive ways for brain signal recording. In general, the brain signals recorded by non-invasive BCIs such as electroencephalography (EEG) is of poor quality with low signal-to-noise ratio, so non-invasive BCIs cannot decode speech with acceptable performance at present. Differently, invasive BCIs such as electrocorticography (ECoG) and stereo-electroencephalography (SEEG) can collect high-quality intracranial brain signals with adequate spatial and temporal resolution, so it is promising to use invasive BCIs to decode speech.Through overt speech using invasive BCIs (ECoG and SEEG) have been developed rapidly and many excellent results are generated in recent years, the covert (imagined) speech decoding is still challenging. There are several reasons behind this situation. The first and major reason is because the associated neural signals are weak and variable compared to overt speech, hence it is very difficult to decode covert speech by machine learning algorithms The second reason is the limited imagined speech dataset recorded invasively. This limited dataset cannot be relieved using an animal model because animals use a system of communication that is believed to be limited to expression of a finite number of utterances that is mostly determined genetically. In addition, recordings on humans are generally restricted to non-invasive techniques. Intracranial data can only be obtained in a clinical environment from patients with drug resistant epilepsy or other neural related conditions. Inclusion criteria, such as normal cognition and the ability to articulate, for speech study further decrease the number of potential subjects. This project (dSPEECH) aims to make some breakthrough regarding the above factors and do a novel study. In dSPEECH, we are ambitious to decode covert speech using invasive BCIs (ECoG and SEEG). We will establish new paradigms for a new generation of brain-to-text BCIs, develop advanced machine learning/deep learning algorithms for decoding covert speech, and construct the world's first ECoG/SEEG dataset for covert speech. We will also tackle the possible problems on ethical issues and data management. With the available of such ECoG/SEEG dataset and the proper decoding methods, we are confident to make big progress on research of decoding covert speech. dSPEECH is a joint project that comprises of multidisciplinary members including researchers from neural engineering and neurosurgeons from clinical medicine. We also have got strong support from partners including famous BCI companies and oversea experienced neurosurgeons. Based on the close and solid collaboration, we believe the world-leading results can be generated in dSPEECH.