The Computations of the Composing Brain: Cross-modal Generality and Computational Specificity

人类认知最令人印象深刻的方面之一是，我们能够产生和理解无限范围的表达，这些表达是从我们长期记忆中的有限词汇项集合中构建的。虽然合成是人类语言的一个定义性特征，但它的神经生物学在很大程度上仍然没有特征。 具体来说，目前的模型缺乏关于各种“组合大脑区域”执行的特定功能的计算细节。“因此，大脑受损的句子处理问题的康复策略也是类似的，而不是针对特定的计算。第二，虽然人们对理解过程中的合成已经研究了几十年，但我们对语言产生过程中的合成几乎一无所知，这主要是由于方法上的原因：说话过程中的运动伪像几乎会损害所有类型的大脑数据。在美国国家科学基金会的资助下，Liina Pylkkänen博士和她的研究团队将对语言成分的神经生物学进行首次全面调查，同时努力实现计算特异性，评估理论语言学中出现的特定功能假设，以及跨模态的一般性，系统地测试理解和生产中的相同刺激。为了克服研究生产的方法障碍，这项研究使用了简短的组合表达式（例如，“灰猫”或“吃肉”），这可以在发音开始之前完全计划好。在这个无人为因素的计划阶段，脑磁图将以毫秒为单位监测大脑活动，脑磁图提供了无与伦比的空间和时间分辨率组合。 该项目主要使用非常简单的单词-图片匹配任务，这对于将来在临床人群和儿童中的应用来说是足够容易的。这项工作的一个具体重点是左前颞叶（LATL）的计算，这是一个在语义痴呆症中萎缩的大脑区域，一种以语义记忆丧失为特征的退行性疾病。该项目将调查LATL在组成方面的作用，以及在语义记忆方面的作用，是否是神经上不同的或语义规范的统一过程的实例。这些发现可能会改变我们对健康大脑和痴呆症患者的组成和语义记忆的理解。

One of the most impressive aspects of human cognition is our ability to produce and comprehend an infinite range of expressions, built from a finite set of lexical items in our long-term memory. Although composition is a defining characteristic of human language, its neurobiology is still largely uncharacterized. Specifically, current models lack computational detail as to the specific functions executed by various "combinatory brain regions." Consequently, rehabilitation strategies for brain damaged individuals suffering sentence processing problems are similarly generic, not targeting specific computations. Second, while composition during comprehension has been studied for decades, we know almost nothing about composition in language production, mostly due to methodological reasons: movement artifacts during talking compromise almost every type of brain data. With funding from the National Science Foundation, Dr. Liina Pylkkänen and her research team will carry out the first comprehensive investigation of the neurobiology of linguistic composition that simultaneously strives both for computational specificity, assessing specific functional hypotheses emerging from theoretical linguistics, as well as cross-modal generality, systematically testing the same stimuli across both comprehension and production. To overcome the methodological hurdles of studying production, this research uses short combinatory expressions (e.g., 'grey cat' or 'eats meat'), which can be fully planned before articulation commences. Brain activity will be monitored millisecond by millisecond during this artifact-free planning stage with magnetoencephalography, which offers an unparalleled combination of spatial and temporal resolution. This project primarily uses a very simple word-picture matching task, which is easy enough for future application in clinical populations and children. A specific focus of this work is the computations of the left anterior temporal lobe (LATL), a brain region that is atrophied in semantic dementia, a degenerative disease characterized by loss of semantic memory. The project will investigate whether the roles of the LATL in composition on the one hand, and in semantic memory on the other, are neurally distinct or instances of a unified process of semantic specification. These findings could be transformative for our understanding of both composition and semantic memory, in the healthy brain as well as in dementia.