Hierarchical Organization of Distributed Semantic Knowledge in the Human Language System

人类语言系统中单词和概念的表征涉及不同层次的抽象，从口语和书面单词的特定情态表征到前颞叶中的抽象概念表征（也可以从其他非语言输入域（包括视觉或触觉）访问）。然而，除了对其定位的深入了解之外，尽管在心理学上进行了数十年的研究，我们对人类思想和大脑中词汇和概念语义表征（层次结构）的本质的计算明确知识仍然是粗糙的，而且往往是描述性的。在阅读和听口语过程中获得的词汇语义表征之间是否存在差异，或者所有的词汇语义表征都是模态独立的？更高层次的概念语义知识是如何表示的？词语义的嵌入模型——从大型文本语料库中的词共现现象中自动导出——已被证明可以很好地解释语言任务中测量的大脑数据。但是，超越共现的语义关系——比如类别和特定范例之间的层次关系，或者不同层次的语义抽象——是如何在这样的分布空间中表现出来的呢？现代人工智能（AI）系统在许多应用领域表现出色，包括自然语言处理和视觉对象识别。然而，同样清楚的是，它们执行这些任务的方式与人类非常不同，一个明显的区别是，它们不像人类的思想和大脑那样表示语义概念。该项目将神经成像实验和人工智能建模相结合，探索不同层次的单词和概念含义如何在人工智能模型中表示并在大脑中编码，特别关注包括颞叶在内的语言系统。结合其他ARENA项目，我们将研究在语义嵌入训练过程中考虑模态独立性和领域通用性是否可以产生更好的模型。最后，本研究将探讨在不同的任务需求下，语义表征是否会动态调整。

The representation of words and concepts in the human language system involves different levels of abstraction ranging from modality-specific representations of spoken vs. written words up to abstract concept representations in the anterior temporal lobe that can also be accessed from other, non-linguistic input domains including vision or touch. However, beyond insights into their localization and despite decades of research in psychology, our computationally explicit knowledge about the nature of such (hierarchies of) lexical and conceptual semantic representations in the human mind and brain still remains coarse and often descriptive. Are there differences between lexical-semantic representations accessed during reading vs. listening to spoken language, or are all lexical-semantic representations modality-independent? How is higher-level conceptual-semantic knowledge represented? Embedding models of word semantics - derived automatically from word co-occurrences in large text corpora - have been shown to account well for brain data measured during language tasks. But how are semantic relationships beyond co-occurrence - like hierarchical relationships between categories and specific exemplars or different levels of semantic abstraction - represented in such distributional spaces? Modern artificial intelligence (AI) systems perform remarkably well in many domains of application - including natural language processing and visual object recognition. It is however also clear that they perform these tasks very differently from the way humans do, and one obvious difference is that they do not represent semantic concepts in the same way as the human mind and brain. This project combines neuroimaging experiments and AI modeling to explore how hierarchically different levels of word and concept meaning are represented in AI models and encoded in the brain, with a particular focus on language systems including the temporal lobes. In combination with other ARENA projects, we will investigate whether taking into account modality-independence and domain-generality during training of semantic embeddings can lead to better models. Lastly, this project will explore whether semantic meaning representations are dynamically adjusted under different task demands.