权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Brain Commands and Beyond: Decoding Inner Speech for Neural Prosthetics

大脑命令及其他：解码神经修复术的内部语音

基本信息

批准号：
MR/X00757X/1
负责人：
金额：
$ 172.23万
依托单位：
University of Oxford
依托单位国家：
英国
项目类别：
Fellowship
财政年份：
2023
资助国家：
英国
起止时间：
2023 至无数据
项目状态：
未结题

来源：
https://gtr.ukri.org/projects?ref=MR%2FX00757X%2F1
关键词：
Brain Commands Beyond Decoding Inner

项目摘要

Millions of people worldwide are deprived of the simple ability to speak because of neurological disorders such as traumatic brain injury, brainstem stroke, or motor neurone disease. In the latter case, the loss of speech is often considered the worst outcome of disease progression. The current state of assistive communication technologies (such as those used by Stephen Hawking) can provide some relief. However, they require residual motor control such as cheek or eye movements. Current technologies also suffer from frustratingly low latencies, with users producing only 20 words per minute. Natural speech, by contrast, is produced at the rate of hundreds of words per minute. For all of these reasons, a new class of speech neuroprosthetic - capable of reading out (or "decoding") intended speech directly from the brain - would provide significant benefits to some of the most isolated people in society. The perfection of speech neuroprosthetics will also represent a scientific milestone in our understanding of how speech and language are represented in the brain. The first speech neuroprosthetic was achieved in a paralysed (anarthric) patient in the summer of 2021. Like much recent work, this landmark study used data from electrodes implanted in the sensorimotor cortex. Although there are advantages to such data, they also have limitations beyond the risk of surgery and installation of electronics into the brain. It is very difficult to obtain large amounts of these surgical data, which limits our ability to leverage the power of deep learning. Another important limitation of surgical data is that speech neuroprosthetics focus on decoding "inner" speech. Unlike overt speech, much less is known about the underlying neurobiology of inner speech. Is it more like imagined articulation ("motor imagery") or imagined audition ("auditory imagery")? Surgical data often targets the sensorimotor cortex, which makes sense for the decoding of overtly articulated speech. But this may be suboptimal for decoding inner speech.Here, we focus on non-invasive inner speech decoding with MRI and magnetoencephalography (MEG). Non-invasive neuroimaging provides, at least, complementary insights to surgical data. The first objective of the project thus seeks to address questions about the nature of inner speech: Where in the brain can we decode it? Does the neural organisation of inner speech differ between individuals? How well can decoders be transferred from one person to another? Answering questions like these will help to design better neuroprosthetics in any imaging modality. Turning to the second objective, there are good reasons to believe that non-invasive methods will produce a viable and less risky speech neuroprosthetic for paralysed patients. MEG-based decoders for speech comprehension (i.e. listening to speech) produce impressive results. Decoding inner speech is harder but - as our pilot data suggests - can be overcome by a combination of big data and deep learning. Thus the project aims to acquire a MEG dataset of sufficient scope (hundreds of hours) within-subject to show that inner speech decoders can, in principle, solve a sequence of tasks from keyword spotting (easier) to large-vocabulary continuous inner speech decoding (harder). The goal is not only to produce state-of-the-art results for each of these tasks, staggered by increasing difficulty and usefulness, but to shape a clear set of objectives for the community to optimise. Thus the MEG data will be released as part of a machine learning competition, inspired by the role that the ImageNet competitions have had in driving the field of computer vision over the past 10 years. We aim to drive similar advances for inner speech decoding.

全世界有数百万人由于神经系统疾病，如创伤性脑损伤、脑干中风或运动神经元疾病而被剥夺了说话的能力。在后一种情况下，言语丧失通常被认为是疾病进展的最坏结果。目前的辅助通信技术（如斯蒂芬霍金使用的技术）可以提供一些缓解。然而，它们需要残余的运动控制，例如脸颊或眼睛运动。目前的技术还受到令人沮丧的低词汇量的困扰，用户每分钟只能产生20个单词。相比之下，自然语言的产生速度是每分钟数百个单词。基于上述原因，一种新型的语言神经假体--能够直接从大脑中阅读（或“解码”）预期的语言--将为社会上一些最孤立的人带来巨大的好处。语言神经修复术的完善也将是我们理解语言和语言在大脑中是如何表现的科学里程碑。2021年夏天，在一名瘫痪（无关节）患者身上实现了第一个语音神经假体。像最近的许多工作一样，这项具有里程碑意义的研究使用了植入感觉运动皮层的电极的数据。虽然这些数据有很多优点，但除了手术和将电子设备安装到大脑中的风险之外，它们也有局限性。很难获得大量的这些手术数据，这限制了我们利用深度学习的能力。手术数据的另一个重要限制是语言神经修复术专注于解码“内部”语言。与外显言语不同，人们对内在言语的神经生物学基础知之甚少。它更像是想象的发音（“运动意象”）还是想象的听觉（“听觉意象”）？手术数据通常针对感觉运动皮层，这对于解码明显清晰的语音是有意义的。但是，这可能是次优解码内部语音。在这里，我们专注于非侵入性内部语音解码与MRI和脑磁图（MEG）。非侵入性神经成像至少为手术数据提供了补充性见解。因此，该项目的第一个目标是解决有关内部言语性质的问题：我们在大脑的什么地方可以解码它？个体之间内部言语的神经组织是否存在差异？解码器能在多大程度上从一个人转移到另一个人？像这样的问题将有助于设计更好的神经假体在任何成像方式。谈到第二个目标，有充分的理由相信，非侵入性的方法将产生一个可行的和风险较小的瘫痪患者的语音神经假体。基于MEG的语音理解解码器（即听语音）产生了令人印象深刻的结果。解码内部语言更难，但正如我们的试点数据所表明的那样，可以通过大数据和深度学习的结合来克服。因此，该项目的目标是在受试者内获得足够范围（数百小时）的MEG数据集，以表明内部语音解码器原则上可以解决从关键字定位（更容易）到大词汇量连续内部语音解码（更难）的一系列任务。我们的目标不仅是为这些任务中的每一个产生最先进的结果，随着难度和实用性的增加而交错，而且要为社区制定一套明确的目标来优化。因此，MEG数据将作为机器学习竞赛的一部分发布，其灵感来自于ImageNet竞赛在过去10年中在推动计算机视觉领域方面的作用。我们的目标是推动内部语音解码的类似进展。