CDI-TYPE II: From Language to Neural Representations of Meaning

CDI-TYPE II：从语言到意义的神经表征

• 批准号: 0835797
• 负责人: Tom Mitchell
• 金额: $ 210万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0835797&HistoricalAwards=false
• 关键词: CDI; TYPE; II; Language; Neural

This project seeks to develop a new understanding of how the brain represents and manipulates meaning, by bringing together the perspectives of brain imaging, machine learning and computational modeling, using converging approaches from behavioral psychology, linguistics, computer science and neuroscience. In particular, the brain activity that encodes the meanings of words, phrases and sentences is studied, along with how the brain encodes the meaning of individual words in terms of their component semantic features, how it modifies its encoding of an individual word when it occurs within a phrase or clause, and how it constructs the encoding of a phrase or clause from the encodings of its component words. This work builds on recent research showing (1) that repeatable patterns of fMRI activation are associated with viewing nouns describing concrete objects such as "hammer" or "toe," (2) that the neural patterns that encode the meanings of these words are similar across different people, and (3) that these encodings are similar whether the person views a word or a picture of the object. Whereas previous work has focused on the neural representation of single words in isolation, this project studies multiple word phrases and sentences, which comprise larger units of knowledge; for example how the neural encoding of a noun is influenced by its adjective (e.g., "fast rabbit" vs. "cuddly rabbit") and how the neural encoding of a proposition is related to the encodings of its component words (how "cut" and "surgeons" combine in the proposition "surgeons cut"). To address these questions, computational models are developed using a diverse set of training data including fMRI data, data from a trillion-word corpus of text that represents typical language use, and behavioral data from language comprehension and judgment tasks, as well as online linguistic knowledge bases such as VerbNet, and theoretical proposals from the cognitive neuroscience literature regarding how and where the brain encodes meaning. These perspectives are integrated into a theory in the form of a computational model trained from diverse data and prior knowledge, and capable of making experimentally testable predictions about the neural encodings and behavioral responses associated with tens of thousands of words, and hundreds of thousands of phrases and sentences.This project potentially constitutes a significant scientific advance in understanding the relation between brain and mind, impacting a variety of scientific disciplines involved in the study of semantics, including linguistics, psychology, philosophy and cognitive science. A second impact comes from use of the methods and results to understand brain pathologies that involve language disturbances, such as aphasia, dyslexia, and autism. A third impact comes from the development of new statistical machine learning algorithms for analyzing and modeling cross-domain data sets to aid in scientific discovery. Finally, the emerging results and methods will have an educational impact through courses on Brain Imaging, Machine Learning, and Psychology taught by the Principal Investigators, and through a new course to be developed specifically on the topic of "Neural representations of meaning," with materials to be made available on the web.

该项目旨在通过汇集大脑成像，机器学习和计算建模的观点，使用行为心理学，语言学，计算机科学和神经科学的融合方法，对大脑如何表示和操纵意义进行新的理解。特别是，大脑的活动，编码的单词，短语和句子的意义进行了研究，随着沿着大脑如何编码的意义，个别单词的组成部分的语义特征，它如何修改其编码的个别单词时，它出现在一个短语或子句，以及它如何构建一个短语或子句的编码从其组成词的编码。这项工作建立在最近的研究基础上，这些研究表明：（1）功能磁共振成像激活的可重复模式与观察描述具体物体的名词（如“锤子”或“脚趾”）有关;（2）编码这些单词含义的神经模式在不同的人中是相似的;（3）无论人们看到的是单词还是物体的图片，这些编码都是相似的。虽然以前的工作集中在孤立的单个单词的神经表征上，但这个项目研究了多个单词短语和句子，它们构成了更大的知识单元;例如，名词的神经编码如何受到其形容词的影响（例如，“快兔子”对“可爱的兔子”）以及命题的神经编码如何与其组成词的编码相关（在命题“外科医生切”中“切”和“外科医生”如何联合收割机）。为了解决这些问题，计算模型是使用不同的训练数据集开发的，包括功能磁共振成像数据，来自代表典型语言使用的万亿字文本语料库的数据，来自语言理解和判断任务的行为数据，以及在线语言知识库，如VerbNet，以及认知神经科学文献中关于大脑如何以及在何处编码意义的理论建议。这些观点被整合到一个理论中，该理论以一个从不同数据和先验知识训练的计算模型的形式存在，并且能够对与数万个单词和数十万个短语和句子相关的神经编码和行为反应做出实验可检验的预测。该项目可能构成理解大脑和心灵之间关系的重大科学进步，影响了语义学研究所涉及的各种科学学科，包括语言学、心理学、哲学和认知科学。第二个影响来自于使用这些方法和结果来理解涉及语言障碍的大脑病理，如失语症，阅读障碍和自闭症。第三个影响来自新的统计机器学习算法的开发，用于分析和建模跨领域数据集，以帮助科学发现。最后，新兴的结果和方法将通过主要研究者教授的脑成像，机器学习和心理学课程产生教育影响，并通过专门针对“意义的神经表征”主题开发的新课程，并在网上提供材料。