Decoding inner speech: An AI approach to transcribing thoughts via EEG & EMG

摘要 丧失通过语言进行交流的能力会对一个人的 自主性、社会交往、职业、心理健康和整体生活质量。许多人失去了 有说话和写作的能力，但要保持思维完整。 内在言语是内在的、故意产生的、非发音的言语思维(例如，阅读 静默)。大脑语言相关区域激活模式的变化与内心语言同时发生 并且可以用脑电(EEG)检测到。此外，虽然内心的言辞不会导致任何 可辨别的声音或清晰度，在发音时同时出现的低幅度电放电 可以用肌电(EMG)检测肌肉。关于正在进行的内在言语的信息被反映 在电生理中，信号(脑电和肌电)可以用来将内心的语言转录成文本或语音。 机器学习算法已经用于这一目的，然而，由此产生的系统具有较低的 准确性和/或受制于非常小的词汇量(~10个单词)。此外，这些系统需要 为每个用户重新培训，这显著增加了个人数据收集时间。的发展。 即用型/最小训练(微调)系统需要大的训练数据集，算法可以使用这些数据集来 学习能够在个人之间传递的高级功能。不幸的是，到目前为止还没有 足够大以训练这些系统的可用数据集。 为了解决这些问题，我组建了一个由耶鲁谷歌人工智能的多学科合作者组成的团队 语言学和耶鲁精神病学将开发一种最先进的深度神经网络来转录内心语言 使用脑电和肌电信号的文本。这个系统将结合人工智能领域的一些最新进展。 由谷歌人工智能开发的智能和数据处理。它将被设计成转录音素，因此，在 原则上，将能够转录任何单词。此外，我们将收集最大的(x120倍)多个主题 (n=150)到目前为止的电生理(EEG+EMG)内部语音数据集(300小时。总共)来训练第一个准备好的- 使用/最少培训的内部语音转录器系统。 这项研究产生的技术有可能从根本上提高人们的生活质量 通过为数以千计的患者提供一种快速交流他们的语言想法的方法。 此外，通过将该系统与目前可用的许多文本到语音转换人工智能之一相结合，我们的 该系统可能会恢复患者产生会话语音的能力。

ABSTRACT Losing the capacity to communicate through language has a significant negative impact on a person’s autonomy, social interactions, occupation, mental health, and overall quality of life. Many people lose the capacity to speak and write but keep their thinking intact. Inner speech is internally and willfully generated, non-articulated verbal thoughts (e.g., reading in silence). Changes in the activation patterns of the brain’s language-related areas co-occur with inner speech and can be detected with electroencephalography (EEG). Furthermore, while inner speech doesn’t lead to any discernible voice sound or articulation, co-occurring low amplitude electrical discharges in the articulatory muscles can be detected with electromyography (EMG). The information about ongoing inner speech reflected in electrophysiological signals (EEG and EMG) can be used to transcribe inner speech into text or voice. Machine learning algorithms have been used for this purpose, however, the resulting systems have low accuracy and/or are constrained by very small vocabularies (~10 words). Furthermore, these systems need to be trained anew for each user, which significantly increases individual data-collection time. The development of ready-to-use/minimal-training (fine tuning) systems requires large training datasets that algorithms can use to learn high-level features capable of being transferred between individuals. Unfortunately, to date there are no available datasets that are large enough to train these systems. To tackle these issues, I have assembled a multidisciplinary team of collaborators from Google AI, Yale linguistics, and Yale Psychiatry to develop a state-of-the-art deep neural network to transcribe inner speech to text using EEG and EMG signals. This system will incorporate some of the latest advances in artificial intelligence and data processing developed by Google AI. It will be designed to transcribe phonemes, thus, in principle, will be able to transcribe any word. Furthermore, we will collect the largest (x120 times) multi-subject (n=150) electrophysiological (EEG+EMG) inner speech dataset to date (300 hrs. in total) to train the first ready- to-use/minimal-training inner speech transcriber system. The technology resulting from this study has the potential to radically improve the quality of life of thousands of patients by providing them with a fast method of communicating their verbal thoughts. Furthermore, by combining this system with one of the many text-to-speech AIs that are currently available, our system could potentially restore the patients’ capacity to produce conversational speech.