Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations
Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations
复制标题
DOI：
10.1111/cogs.13291
复制
发表时间：
2023-05-01
影响因子：
2.5
通讯作者：
McRae，Ken
中科院分区：
文献类型：
--
作者：
Brown，Kevin S.;Yee，Eiling;McRae，Ken
分布式语义模型是从语料库中提取语义信息的主要方法。然而，一个关键的问题仍然存在：什么类型的词之间的语义关系DSM检测？先前的工作通常使用受限于语义相似性和/或一般语义相关性的有限人类数据来解决这个问题。我们测试了当前认知和心理语言学研究中流行的八种DSM（正逐点互信息;全局向量;以及使用词，上下文和均值嵌入的Skip‐gram和连续词袋（CBOW）的三种变体），这些DSM基于理论动机，丰富的语义关系集，涉及来自多个句法类的词，并跨越抽象-具体连续体（19组评级）。我们发现，总体而言，DSM最擅长捕捉整体语义相似性，也可以捕捉在句子理解中起重要作用的动词-名词主题角色关系和名词-名词事件关系。有趣的是，Skip‐gram和CBOW在捕捉相似性方面表现最好，而GloVe在主题角色和基于事件的关系方面占主导地位。我们讨论了我们的研究结果的理论和实践意义，为这些模型的用户提出建议，并展示了基于事件的关系模型性能的显着差异。
Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relatedness. We tested eight DSMs that are popular in current cognitive and psycholinguistic research (positive pointwise mutual information; global vectors; and three variations each of Skip‐gram and continuous bag of words (CBOW) using word, context, and mean embeddings) on a theoretically motivated, rich set of semantic relations involving words from multiple syntactic classes and spanning the abstract–concrete continuum (19 sets of ratings). We found that, overall, the DSMs are best at capturing overall semantic similarity and also can capture verb–noun thematic role relations and noun–noun event‐based relations that play important roles in sentence comprehension. Interestingly, Skip‐gram and CBOW performed the best in terms of capturing similarity, whereas GloVe dominated the thematic role and event‐based relations. We discuss the theoretical and practical implications of our results, make recommendations for users of these models, and demonstrate significant differences in model performance on event‐based relations.