Interpretable Entity Representations through Large-Scale Typing

Interpretable Entity Representations through Large-Scale Typing
复制标题

DOI：
10.18653/v1/2020.findings-emnlp.54
复制
发表时间：
2020-04
期刊：
ArXiv
影响因子：
--
通讯作者：
Yasumasa Onoe;Greg Durrett
Yasumasa Onoe;Greg Durrett
中科院分区：
其他
文献类型：
--
作者：
Yasumasa Onoe;Greg Durrett

文献摘要

被引文献

相似文献

在自然语言处理的标准方法中，文本中的实体通常嵌入具有预训练模型的密集向量空间中。以这种方式产生的嵌入在输入到下游模型时是有效的，但它们需要在任务结束时进行微调，并且从根本上难以解释。在本文中，我们提出了一种方法来创建人类可读的实体表示，并在与实体相关的任务上实现开箱即用的高性能。我们的表示是向量，其值对应于细粒度实体类型的后验概率，表明类型模型对实体属于相应类型的决策的置信度。我们使用细粒度实体类型模型获得这些表示，该模型可以在有监督的超细实体类型数据（Choi et al. 2018）或来自维基百科的远程监督示例上进行训练。在涉及识别实体身份的实体探测任务中，我们在无参数下游模型中使用的嵌入与训练模型中基于ELMo和bert的嵌入具有竞争性能。我们还表明，对于特定领域，以基于学习的方式减少类型集的大小是可能的。最后，我们证明了这些嵌入可以通过少量规则进行事后修改，以纳入领域知识并提高性能。
In standard methodology for natural language processing, entities in text are typically embedded in dense vector spaces with pre-trained models. The embeddings produced this way are effective when fed into downstream models, but they require end-task fine-tuning and are fundamentally difficult to interpret. In this paper, we present an approach to creating entity representations that are human readable and achieve high performance on entity-related tasks out of the box. Our representations are vectors whose values correspond to posterior probabilities over fine-grained entity types, indicating the confidence of a typing model’s decision that the entity belongs to the corresponding type. We obtain these representations using a fine-grained entity typing model, trained either on supervised ultra-fine entity typing data (Choi et al. 2018) or distantly-supervised examples from Wikipedia. On entity probing tasks involving recognizing entity identity, our embeddings used in parameter-free downstream models achieve competitive performance with ELMo- and BERT-based embeddings in trained models. We also show that it is possible to reduce the size of our type set in a learning-based way for particular domains. Finally, we show that these embeddings can be post-hoc modified through a small number of rules to incorporate domain knowledge and improve performance.