Abstractive Document Summarization with Word Embedding Reconstruction
Abstractive Document Summarization with Word Embedding Reconstruction
复制标题
DOI：
10.26615/978-954-452-072-4_178
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Jingyi You;Chenlong Hu;Hidetaka Kamigaito;Hiroya Takamura;M. Okumura
中科院分区：
文献类型：
--
作者：
Jingyi You;Chenlong Hu;Hidetaka Kamigaito;Hiroya Takamura;M. Okumura
神经序列到序列（Seq 2Seq）模型和BERT分别在无预训练和有预训练的抽象文档摘要（ADS）方面取得了实质性的改进。然而，他们有时会反复注意不重要的源短语，而错误地忽略重要的。我们提出了两个层面上的重建机制，以缓解这个问题。序列级重构器从目标摘要的隐藏层重构整个文档，而词嵌入级重构器在目标端重构源的词嵌入的平均值，以保证尽可能多的关键信息被包括在摘要中。基于逆文档频率（IDF）衡量单词重要性的假设，我们进一步在嵌入级重建器中利用IDF权重。拟议的框架导致有希望的改进ROUGE指标和人类评级的CNN/每日邮报和新闻编辑室摘要数据集。
Neural sequence-to-sequence (Seq2Seq) models and BERT have achieved substantial improvements in abstractive document summarization (ADS) without and with pre-training, respectively. However, they sometimes repeatedly attend to unimportant source phrases while mistakenly ignore important ones. We present reconstruction mechanisms on two levels to alleviate this issue. The sequence-level reconstructor reconstructs the whole document from the hidden layer of the target summary, while the word embedding-level one rebuilds the average of word embeddings of the source at the target side to guarantee that as much critical information is included in the summary as possible. Based on the assumption that inverse document frequency (IDF) measures how important a word is, we further leverage the IDF weights in our embedding-level reconstructor. The proposed frameworks lead to promising improvements for ROUGE metrics and human rating on both the CNN/Daily Mail and Newsroom summarization datasets.