A Unified Encoder-Decoder Framework with Entity Memory
A Unified Encoder-Decoder Framework with Entity Memory
复制标题
DOI：
10.48550/arxiv.2210.03273
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Zhihan Zhang;W. Yu;Chenguang Zhu;Meng Jiang
中科院分区：
文献类型：
--
作者：
Zhihan Zhang;W. Yu;Chenguang Zhu;Meng Jiang
实体作为真实世界知识的重要载体，在许多自然语言处理任务中起着关键的作用，本文重点研究如何将实体知识融入到一个编码器-解码器框架中，以生成信息文本。现有的方法试图索引，检索和读取外部文档作为证据，但他们遭受了巨大的计算开销。在这项工作中，我们提出了一个编码器-解码器框架与实体内存，即EDMem。实体知识作为潜在表示存储在存储器中，并且存储器在维基百科上与编码器-解码器参数一起进行沿着预训练。为了精确地生成实体名称，我们设计了三种解码方法，通过链接内存中的实体来约束实体生成。EDMem是一个统一的框架，可用于各种实体密集型问题回答和生成任务。大量的实验结果表明，EDMem优于基于内存的自动编码器模型和非内存编码器-解码器模型。
Entities, as important carriers of real-world knowledge, play a key role in many NLP tasks.We focus on incorporating entity knowledge into an encoder-decoder framework for informative text generation. Existing approaches tried to index, retrieve, and read external documents as evidence, but they suffered from a large computational overhead. In this work, we propose an encoder-decoder framework with an entity memory, namely EDMem. The entity knowledge is stored in the memory as latent representations, and the memory is pre-trained on Wikipedia along with encoder-decoder parameters. To precisely generate entity names, we design three decoding methods to constrain entity generation by linking entities in the memory. EDMem is a unified framework that can be used on various entity-intensive question answering and generation tasks. Extensive experimental results show that EDMem outperforms both memory-based auto-encoder models and non-memory encoder-decoder models.