Hierarchical Organization of Distributed Semantic Knowledge in the Human Language System

词汇和概念在人类语言系统中的表征涉及不同程度的抽象，从口语和书面语的特定通道表征到前颞叶的抽象概念表征，这些抽象概念表征也可以从其他非语言输入领域获得，包括视觉或触觉。然而，除了对它们的局限性的洞察之外，尽管我们在心理学上进行了数十年的研究，但我们对人类大脑中这种词汇和概念语义表征的性质(层次结构)的计算显性知识仍然粗略，往往是描述性的。在阅读和听口语过程中获得的词汇-语义表征之间是否存在差异，或者所有的词汇-语义表征都是独立于通道的？更高层次的概念语义知识是如何表示的？词汇语义的嵌入模型--从大型文本语料库中的词汇共现中自动派生出来--已被证明能很好地解释语言任务期间测量的大脑数据。但是，在这样的分布空间中，超越共现的语义关系--如类别和特定样本之间的层次关系或不同级别的语义抽象--是如何表示的呢？现代人工智能(AI)系统在许多应用领域表现出色-包括自然语言处理和视觉对象识别。然而，很明显，它们执行这些任务的方式与人类的方式非常不同，一个明显的区别是，它们不像人类的头脑和大脑一样代表语义概念。该项目结合神经成像实验和人工智能建模，探索不同层次的单词和概念意义如何在人工智能模型中表示并在大脑中编码，特别关注包括颞叶在内的语言系统。结合其他Arena项目，我们将调查在语义嵌入的训练过程中考虑到通道独立性和领域通用性是否可以产生更好的模型。最后，本研究将探讨语义表征是否会根据不同的任务需求而动态调整。

The representation of words and concepts in the human language system involves different levels of abstraction ranging from modality-specific representations of spoken vs. written words up to abstract concept representations in the anterior temporal lobe that can also be accessed from other, non-linguistic input domains including vision or touch. However, beyond insights into their localization and despite decades of research in psychology, our computationally explicit knowledge about the nature of such (hierarchies of) lexical and conceptual semantic representations in the human mind and brain still remains coarse and often descriptive. Are there differences between lexical-semantic representations accessed during reading vs. listening to spoken language, or are all lexical-semantic representations modality-independent? How is higher-level conceptual-semantic knowledge represented? Embedding models of word semantics - derived automatically from word co-occurrences in large text corpora - have been shown to account well for brain data measured during language tasks. But how are semantic relationships beyond co-occurrence - like hierarchical relationships between categories and specific exemplars or different levels of semantic abstraction - represented in such distributional spaces? Modern artificial intelligence (AI) systems perform remarkably well in many domains of application - including natural language processing and visual object recognition. It is however also clear that they perform these tasks very differently from the way humans do, and one obvious difference is that they do not represent semantic concepts in the same way as the human mind and brain. This project combines neuroimaging experiments and AI modeling to explore how hierarchically different levels of word and concept meaning are represented in AI models and encoded in the brain, with a particular focus on language systems including the temporal lobes. In combination with other ARENA projects, we will investigate whether taking into account modality-independence and domain-generality during training of semantic embeddings can lead to better models. Lastly, this project will explore whether semantic meaning representations are dynamically adjusted under different task demands.