CRCNS Research Proposal: Novel computational approaches for neural speech prostheses and causal dynamics of language processing

CRCNS 研究提案：神经语音假体和语言处理因果动力学的新型计算方法

• 批准号: 2309057
• 负责人: Yao Wang
• 金额: $ 95万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2309057&HistoricalAwards=false
• 关键词: CRCNS; Research; Proposal; Novel; computational

This project aims to develop computational approaches that will allow for a deeper understanding of the neurobiology of language and translate to novel clinical applications for speech and language. The computational research is necessary to enable future speech neuroprostheses, which would allow patients with degenerative conditions or neurological damage to drive a speech synthesizer using their intact cortical structures. Additionally, the combination of tools for analyzing connectivity and regions critical for language in individual brains will provide insight into the network dynamics of language cortex and open the door to replacing the clinical practice of using electrical stimulation to map language-critical regions, which is not well tolerated by all patients.The research is framed across three intertwining thrusts. The first thrust will explore the use of deep learning for decoding produced speech from various neural signals captured by intracranial depth and surface electrodes. This thrust will develop models within and across multiple patients that robustly decode speech while overcoming current limitations in the field, leading to potential integration into speech neuroprostheses. The second thrust will explore efficient algorithms for estimating brain connectivity dynamics from the recorded signals. This thrust will develop novel techniques that estimate directed connectivity among a large number of recording sites, which is essential for understanding the dynamic interactions across cortex during cognitive processing. The third thrust will develop deep-learning models that can predict brain regions that are causally critical for language processing based on the neural recordings alone. These models will serve to pinpoint brain regions that are critical for language processing without needing to electrically stimulate the brain. Taken together, the proposed research leverages recent innovations in deep learning (e.g. transformers, graph neural networks, self-supervised learning) to overcome challenges stemming from the non-structured and varied placements of electrodes across patients, data scarcity, scalability and stability. These new approaches will be shared with the scientific community as open-source and replicable technologies. This project is jointly funded by the Collaborative Research in Computational Neuroscience (CRCNS) program and the Disabilities and Rehabilitation Engineering Program (DARE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在开发计算方法，以更深入地理解语言的神经生物学，并将其转化为语音和语言的新临床应用。计算研究对于未来的语音神经假体是必要的，这将允许患有退行性疾病或神经损伤的患者使用其完整的皮层结构驱动语音合成器。此外，结合分析个体大脑中连接性和语言关键区域的工具，将深入了解语言皮层的网络动态，并为取代使用电刺激来绘制语言关键区域的临床实践打开大门，这并不是所有患者都能很好地耐受的。这项研究是围绕三个相互交织的主题展开的。第一个重点将探索使用深度学习来解码由颅内深度和表面电极捕获的各种神经信号产生的语音。这一推动力将在多个患者体内和跨患者开发模型，这些模型能够鲁棒地解码语音，同时克服该领域目前的局限性，从而有可能集成到语音神经假体中。第二个重点将探索从记录的信号中估计大脑连接动力学的有效算法。这一推力将开发新的技术，估计大量记录站点之间的定向连接，这是理解认知过程中跨皮层的动态相互作用所必需的。第三个重点是开发深度学习模型，可以仅根据神经记录来预测对语言处理至关重要的大脑区域。这些模型将用于精确定位对语言处理至关重要的大脑区域，而无需对大脑进行电刺激。总的来说，拟议的研究利用深度学习的最新创新（例如transformers，graph neural networks，self-supervised learning）来克服患者之间电极的非结构化和不同放置，数据稀缺，可扩展性和稳定性带来的挑战。这些新方法将作为开放源码和可复制的技术与科学界分享。该项目由计算神经科学合作研究（CRCNS）计划和残疾与康复工程计划（DARE）共同资助。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。