Exploiting Structural and Semantic Context for Commonsense Knowledge Base Completion
Exploiting Structural and Semantic Context for Commonsense Knowledge Base Completion
复制标题
利用结构和语义上下文来完成常识知识库
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Yejin Choi
中科院分区：
文献类型：
--
作者：
Chaitanya Malaviya;Chandra Bhagavatula;Antoine Bosselut;Yejin Choi
常识知识图(如原子和概念网)的自动知识库补全与大量研究的传统知识库(如自由库)相比提出了独特的挑战。常识知识图使用自由格式的文本来表示节点，导致与传统的知识库相比，节点多了一个数量级(∼的原子节点是自由库的18倍(FB15K237))。重要的是，这意味着更稀疏的图结构-这是现有知识库完成方法的主要挑战，这些方法假设在相对较小的节点集上密集连接图。在本文中，我们提出了新颖的知识库补全模型，该模型可以通过利用节点的结构和语义上下文来解决这些挑战。具体地说，我们研究了两个关键思想：(1)从局部图结构中学习，使用图卷积网络和自动图致密化；(2)将学习从预先训练的语言模型转移到知识图，以增强知识的上下文表示。我们描述了我们的方法，将来自这两个来源的信息整合到一个联合模型中，并提供了第一个经验结果，用于原子上的知识库完成和概念网上的评估和排名度量。我们的结果证明了语言模型表示在提高链接预测性能方面的有效性，以及当对子图进行计算效率训练时，从局部图结构(概念网的MRR中+1.5点)学习的优势。对模型预测的进一步分析有助于揭示语言模型很好地捕捉到的常识知识的类型。
Automatic KB completion for commonsense knowledge graphs (e.g., ATOMIC and ConceptNet) poses unique challenges compared to the much studied conventional knowledge bases (e.g., Freebase). Commonsense knowledge graphs use free-form text to represent nodes, resulting in orders of magnitude more nodes compared to conventional KBs ( ∼18x more nodes in ATOMIC compared to Freebase (FB15K237)). Importantly, this implies significantly sparser graph structures — a major challenge for existing KB completion methods that assume densely connected graphs over a relatively smaller set of nodes. In this paper, we present novel KB completion models that can address these challenges by exploiting the structural and semantic context of nodes. Specifically, we investigate two key ideas: (1) learning from local graph structure, using graph convolutional networks and automatic graph densification and (2) transfer learning from pre-trained language models to knowledge graphs for enhanced contextual representation of knowledge. We describe our method to incorporate information from both these sources in a joint model and provide the first empirical results for KB completion on ATOMIC and evaluation with ranking metrics on ConceptNet. Our results demonstrate the effectiveness of language model representations in boosting link prediction performance and the advantages of learning from local graph structure (+1.5 points in MRR for ConceptNet) when training on subgraphs for computational efficiency. Further analysis on model predictions shines light on the types of commonsense knowledge that language models capture well.