US-French Collaboration: Collaborative Research: Neuro-Computational Models of Natural Language

美法合作：合作研究：自然语言的神经计算模型

• 批准号: 1607441
• 负责人: John Hale
• 金额: $ 56.98万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1607441&HistoricalAwards=false
• 关键词: US; French; Collaboration; Collaborative; Research

Our society is built upon shared ideas, ideas that get from one person to another via language that is "understood." But how do brains give us the ability to understand a stream of spoken words? This is a grand challenge question in computational neuroscience. This project addresses it using mathematical models of the language understanding process. These models reflect insights from computer science as well as linguistics. They allow investigators to ask: which process model best accounts for the signals from a particular brain region, at particular moment in time? The signals come from people listening to French and English versions of the same book.  By comparing across models and across languages, the project seeks to differentiate between aspects of the understanding process that are language-specific and aspects that might be common to all humans. Increasingly precise modeling of this sort paves the way for future work with individuals who have trouble using language, such as those with Autism Spectrum Disorder. It could also lead to better computer systems, ones that use language in a brain-inspired way.Bringing together computational linguists and cognitive neuroscientists, this project pursues two specific questions: (1) what aspects of sentence structure determine our expectations for upcoming words? and (2) what is the detailed balance between memorization and composition in natural language? Using electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI) the PIs examine participants' neural responses to the spoken recitation of a literary work. These neural signals are fitted by time series predictors, themselves derived from linguistically plausible grammars and other language models. The project explores a family of such models, varying the size of grammatical units as well as the propensity for such units to be simply memorized as opposed to built up, step by step. Via information-theoretical complexity metrics, these theories derive quantitative predictions about the moment-by-moment neural responses of a person hearing a story.  The approach as a whole leads to computationally explicit process models that are grounded in human brain responses to naturalistic text across two languages.A companion project is being funded by the French National Research Agency (ANR).

我们的社会是建立在共享的思想，思想从一个人到另一个通过语言是“理解”。“但是大脑是如何赋予我们理解一连串口语的能力的呢？这是计算神经科学中的一个重大挑战性问题。这个项目使用语言理解过程的数学模型来解决这个问题。这些模型反映了计算机科学和语言学的见解。研究人员可以借此提出这样一个问题：哪种处理模型最能解释在特定时刻来自特定大脑区域的信号？这些信号来自于听同一本书的法语和英语版本的人。通过跨模型和跨语言的比较，该项目试图区分理解过程中特定语言的方面和可能对所有人都通用的方面。这种日益精确的建模为未来与语言使用困难的个人（例如自闭症谱系障碍患者）的合作铺平了道路。这个项目将计算语言学家和认知神经科学家聚集在一起，研究两个具体的问题：（1）句子结构的哪些方面决定了我们对即将出现的单词的预期？（2）自然语言中记忆和写作的平衡点是什么？使用脑电图（EEG）和功能性磁共振成像（fMRI），PI检查参与者对文学作品的口头背诵的神经反应。这些神经信号由时间序列预测器拟合，这些预测器本身来自语言学上合理的语法和其他语言模型。该项目探索了一系列这样的模型，改变了语法单位的大小，以及这些单位的倾向，简单地记忆，而不是建立起来，一步一步。通过信息理论复杂性度量，这些理论可以对一个人听故事时的每时每刻的神经反应进行定量预测。这种方法作为一个整体，可以产生基于人脑对两种语言自然文本反应的计算显式过程模型。法国国家研究机构（ANR）正在资助一个配套项目。