Brain Commands and Beyond: Decoding Inner Speech for Neural Prosthetics

大脑命令及其他：解码神经修复术的内部语音

基本信息

批准号：
MR/X00757X/1
负责人：
金额：
$ 172.23万
依托单位：
University of Oxford
依托单位国家：
英国
项目类别：
Fellowship
财政年份：
2023
资助国家：
英国
起止时间：
2023 至无数据
项目状态：
未结题

来源：
https://gtr.ukri.org/projects?ref=MR%2FX00757X%2F1
关键词：
Brain Commands Beyond Decoding Inner

项目摘要

Millions of people worldwide are deprived of the simple ability to speak because of neurological disorders such as traumatic brain injury, brainstem stroke, or motor neurone disease. In the latter case, the loss of speech is often considered the worst outcome of disease progression. The current state of assistive communication technologies (such as those used by Stephen Hawking) can provide some relief. However, they require residual motor control such as cheek or eye movements. Current technologies also suffer from frustratingly low latencies, with users producing only 20 words per minute. Natural speech, by contrast, is produced at the rate of hundreds of words per minute. For all of these reasons, a new class of speech neuroprosthetic - capable of reading out (or "decoding") intended speech directly from the brain - would provide significant benefits to some of the most isolated people in society. The perfection of speech neuroprosthetics will also represent a scientific milestone in our understanding of how speech and language are represented in the brain. The first speech neuroprosthetic was achieved in a paralysed (anarthric) patient in the summer of 2021. Like much recent work, this landmark study used data from electrodes implanted in the sensorimotor cortex. Although there are advantages to such data, they also have limitations beyond the risk of surgery and installation of electronics into the brain. It is very difficult to obtain large amounts of these surgical data, which limits our ability to leverage the power of deep learning. Another important limitation of surgical data is that speech neuroprosthetics focus on decoding "inner" speech. Unlike overt speech, much less is known about the underlying neurobiology of inner speech. Is it more like imagined articulation ("motor imagery") or imagined audition ("auditory imagery")? Surgical data often targets the sensorimotor cortex, which makes sense for the decoding of overtly articulated speech. But this may be suboptimal for decoding inner speech.Here, we focus on non-invasive inner speech decoding with MRI and magnetoencephalography (MEG). Non-invasive neuroimaging provides, at least, complementary insights to surgical data. The first objective of the project thus seeks to address questions about the nature of inner speech: Where in the brain can we decode it? Does the neural organisation of inner speech differ between individuals? How well can decoders be transferred from one person to another? Answering questions like these will help to design better neuroprosthetics in any imaging modality. Turning to the second objective, there are good reasons to believe that non-invasive methods will produce a viable and less risky speech neuroprosthetic for paralysed patients. MEG-based decoders for speech comprehension (i.e. listening to speech) produce impressive results. Decoding inner speech is harder but - as our pilot data suggests - can be overcome by a combination of big data and deep learning. Thus the project aims to acquire a MEG dataset of sufficient scope (hundreds of hours) within-subject to show that inner speech decoders can, in principle, solve a sequence of tasks from keyword spotting (easier) to large-vocabulary continuous inner speech decoding (harder). The goal is not only to produce state-of-the-art results for each of these tasks, staggered by increasing difficulty and usefulness, but to shape a clear set of objectives for the community to optimise. Thus the MEG data will be released as part of a machine learning competition, inspired by the role that the ImageNet competitions have had in driving the field of computer vision over the past 10 years. We aim to drive similar advances for inner speech decoding.

由于神经系统疾病，例如脑损伤，脑干中风或运动神经元疾病，全世界数百万的人被剥夺了说话的简单能力。在后一种情况下，言语丧失通常被认为是疾病进展的最糟糕结果。辅助通信技术的当前状态（例如斯蒂芬·霍金（Stephen Hawking）使用的技术）可以缓解。但是，它们需要残留的运动控制，例如脸颊或眼睛运动。当前的技术还遭受了令人沮丧的低潜伏期，用户每分钟只能产生20个单词。相比之下，自然语音以每分钟数百个单词的速度产生。由于所有这些原因，可以直接从大脑中读取（或“解码”）意图的言语的新的言语神经假话，这将为社会上一些最孤立的人带来重大好处。言语神经心想的完美也将代表我们对言语和语言在大脑中如何代表的科学里程碑。在2021年夏天，在瘫痪（Anarthric）患者中实现了第一个语音神经假体。与最近的许多工作一样，这项具有里程碑意义的研究使用了植入了感觉运动皮层中的电极的数据。尽管此类数据具有优势，但它们也有超出手术风险和将电子设备安装到大脑中的局限性。获得大量这些手术数据非常困难，这限制了我们利用深度学习力量的能力。手术数据的另一个重要局限性是语音神经po的侧重于解码“内部”语音。与公开的言语不同，关于内部语音的基本神经生物学知之甚少。它更像是想象中的发音（“运动图像”）或想象中的试听（“听觉图像”）？手术数据通常靶向感觉运动皮层，这对于对公开表达的语音的解码很有意义。但这可能是解码内部语音的次优。在这里，我们专注于使用MRI和磁脑摄影（MEG）的非侵入性内部语音解码。非侵入性神经影像学至少提供了对手术数据的互补见解。因此，该项目的第一个目标试图解决有关内部语音本质的问题：我们可以在哪里解码它？个体之间内部语音的神经组织会有所不同吗？解码器如何从一个人转移到另一个人？回答这样的问题将有助于在任何成像模式中设计更好的神经植物。转向第二个目标，有充分的理由相信非侵入性方法会为瘫痪的患者产生可行且风险较小的语音神经假体。基于MEG的语音理解解码器（即聆听语音）产生了令人印象深刻的结果。解码内部语音更难，但是 - 正如我们的飞行员数据所建议的那样，可以通过大数据和深度学习的结合来克服。因此，该项目旨在获取足够范围（数百个小时）内部主体的MEG数据集，以表明内部语音解码器原则上可以解决一系列任务，从关键字发现（易于）到大型vocabulary vocabulary连续的内部语音解码（更难）。该目标不仅是为每个任务中的每一个都产生最新的结果，并因增加难度和实用性而错开，而且要塑造一套明确的目标，以供社区优化。因此，MEG数据将作为机器学习竞赛的一部分发布，灵感来自于ImageNet竞赛在过去10年中推动计算机视野领域的作用。我们旨在推动内部语音解码的类似进步。