CAREER: Uncovering the brain circuitry of language and its interaction with other modalities

职业：揭示语言的大脑回路及其与其他模式的相互作用

• 批准号: 2237064
• 负责人: Leila Wehbe
• 金额: $ 60万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2028-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2237064&HistoricalAwards=false
• 关键词: CAREER; Uncovering; brain; circuitry; language

This project aims to better understand how different brain areas interact with each other to enable understanding and production of language in complex real-life settings. Language is a crucial part of human experience, versatile and intimately connected to other abilities. For example, reading a story relies not only on language processing but also on social reasoning, inference of subsequent events, visual imagery, etc. However, due to the complexity of language, most neuroimaging experiments have focused on one aspect of language (e.g., syntax) and studied language in isolation from other cognitive abilities. This has led to theories about which brain areas are involved in language processing and the broad role of each area, but has not been able to reveal what individual units of meaning are processed in each area, how information flows between areas, and how these areas interact with other areas that process other cognitive abilities such as vision or social reasoning. This proposal brings together more powerful experimental data of language in a natural context and uses machine learning, statistics and advances in artificial intelligence to study real-world language use.This project seeks to further understanding of the neurobiology of language and its interaction with other modalities, while also improving AI methods by jointly modeling brain activity and stimulus. Aim 1 will focus on using intracortical recordings (stereoencephalography) to trace the flow of information in the language system at a fine spatial and temporal resolution, enabling more precise discovery of where language features are processed and how information passes between regions. Aim 2 will connect language to visual and social processing, using extensive functional magnetic resonance imaging (fMRI) data from individuals watching a popular sitcom to model the interactions between the language system and other brain systems. Aim 3 will extend this approach to relating brain activity to behavior. A new fMRI dataset where participants engage in natural conversation will be collected and used to study the underlying high-level brain representations. The proposed multidisciplinary research will lead to new educational material as well as new opportunities for students to explore the use of machine learning and AI for science.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在更好地了解不同的大脑区域如何相互作用，从而在复杂的现实生活环境中理解和产生语言。语言是人类经验的重要组成部分，用途广泛，与其他能力密切相关。例如，阅读故事不仅依赖于语言处理，还依赖于社会推理、后续事件推理、视觉意象等。然而，由于语言的复杂性，大多数神经影像学实验都集中在语言的一个方面（如句法），并将语言与其他认知能力分开研究。这导致了关于哪些大脑区域参与语言处理以及每个区域的广泛作用的理论，但无法揭示每个区域处理的单个意义单位，区域之间的信息如何流动，以及这些区域如何与处理其他认知能力（如视觉或社会推理）的其他区域相互作用。该提案汇集了自然环境中更强大的语言实验数据，并使用机器学习，统计学和人工智能的进步来研究现实世界的语言使用。该项目旨在进一步了解语言的神经生物学及其与其他模式的相互作用，同时通过共同建模大脑活动和刺激来改进人工智能方法。目标1将侧重于使用皮质内记录（立体脑电图）以精细的空间和时间分辨率跟踪语言系统中的信息流，从而更精确地发现语言特征在哪里被处理以及信息如何在区域之间传递。目标2将把语言与视觉和社会处理联系起来，使用大量的功能性磁共振成像（fMRI）数据，这些数据来自观看流行情景喜剧的个体，以模拟语言系统与其他大脑系统之间的相互作用。目标3将把这种方法扩展到将大脑活动与行为联系起来。一个新的fMRI数据集将收集参与者进行自然对话，并用于研究潜在的高级大脑表征。拟议的多学科研究将带来新的教育材料，并为学生探索机器学习和人工智能在科学中的应用提供新的机会。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。