Hierarchical Organization of Distributed Semantic Knowledge in the Human Language System

点击翻译按钮获取中文摘要

The representation of words and concepts in the human language system involves different levels of abstraction ranging from modality-specific representations of spoken vs. written words up to abstract concept representations in the anterior temporal lobe that can also be accessed from other, non-linguistic input domains including vision or touch. However, beyond insights into their localization and despite decades of research in psychology, our computationally explicit knowledge about the nature of such (hierarchies of) lexical and conceptual semantic representations in the human mind and brain still remains coarse and often descriptive. Are there differences between lexical-semantic representations accessed during reading vs. listening to spoken language, or are all lexical-semantic representations modality-independent? How is higher-level conceptual-semantic knowledge represented? Embedding models of word semantics - derived automatically from word co-occurrences in large text corpora - have been shown to account well for brain data measured during language tasks. But how are semantic relationships beyond co-occurrence - like hierarchical relationships between categories and specific exemplars or different levels of semantic abstraction - represented in such distributional spaces? Modern artificial intelligence (AI) systems perform remarkably well in many domains of application - including natural language processing and visual object recognition. It is however also clear that they perform these tasks very differently from the way humans do, and one obvious difference is that they do not represent semantic concepts in the same way as the human mind and brain. This project combines neuroimaging experiments and AI modeling to explore how hierarchically different levels of word and concept meaning are represented in AI models and encoded in the brain, with a particular focus on language systems including the temporal lobes. In combination with other ARENA projects, we will investigate whether taking into account modality-independence and domain-generality during training of semantic embeddings can lead to better models. Lastly, this project will explore whether semantic meaning representations are dynamically adjusted under different task demands.