CRCNS Research Proposal: Novel computational approaches for neural speech prostheses and causal dynamics of language processing

点击翻译按钮获取中文摘要

This project aims to develop computational approaches that will allow for a deeper understanding of the neurobiology of language and translate to novel clinical applications for speech and language. The computational research is necessary to enable future speech neuroprostheses, which would allow patients with degenerative conditions or neurological damage to drive a speech synthesizer using their intact cortical structures. Additionally, the combination of tools for analyzing connectivity and regions critical for language in individual brains will provide insight into the network dynamics of language cortex and open the door to replacing the clinical practice of using electrical stimulation to map language-critical regions, which is not well tolerated by all patients.The research is framed across three intertwining thrusts. The first thrust will explore the use of deep learning for decoding produced speech from various neural signals captured by intracranial depth and surface electrodes. This thrust will develop models within and across multiple patients that robustly decode speech while overcoming current limitations in the field, leading to potential integration into speech neuroprostheses. The second thrust will explore efficient algorithms for estimating brain connectivity dynamics from the recorded signals. This thrust will develop novel techniques that estimate directed connectivity among a large number of recording sites, which is essential for understanding the dynamic interactions across cortex during cognitive processing. The third thrust will develop deep-learning models that can predict brain regions that are causally critical for language processing based on the neural recordings alone. These models will serve to pinpoint brain regions that are critical for language processing without needing to electrically stimulate the brain. Taken together, the proposed research leverages recent innovations in deep learning (e.g. transformers, graph neural networks, self-supervised learning) to overcome challenges stemming from the non-structured and varied placements of electrodes across patients, data scarcity, scalability and stability. These new approaches will be shared with the scientific community as open-source and replicable technologies. This project is jointly funded by the Collaborative Research in Computational Neuroscience (CRCNS) program and the Disabilities and Rehabilitation Engineering Program (DARE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.