Positional Encoding to Control Output Sequence Length
Positional Encoding to Control Output Sequence Length
复制标题
DOI：
10.18653/v1/n19-1401
复制
发表时间：
2019-04
期刊：
影响因子：
--
通讯作者：
Sho Takase;Naoaki Okazaki
中科院分区：
文献类型：
--
作者：
Sho Takase;Naoaki Okazaki
神经编码器-解码器模型在自然语言生成任务中取得了成功。然而，抽象摘要的真实的应用必须考虑生成的摘要不应超过期望长度的附加约束。在本文中，我们提出了正弦位置编码的简单但有效的扩展（Vaswani等人，2017），使得神经编码器-解码器模型保留长度约束。与以前学习长度嵌入的研究不同，该方法可以生成任何长度的文本，即使目标长度在训练数据中不可见。实验结果表明，该方法不仅可以控制生成长度，而且可以提高ROUGE得分。
Neural encoder-decoder models have been successful in natural language generation tasks. However, real applications of abstractive summarization must consider an additional constraint that a generated summary should not exceed a desired length. In this paper, we propose a simple but effective extension of a sinusoidal positional encoding (Vaswani et al., 2017) so that a neural encoder-decoder model preserves the length constraint. Unlike previous studies that learn length embeddings, the proposed method can generate a text of any length even if the target length is unseen in training data. The experimental results show that the proposed method is able not only to control generation length but also improve ROUGE scores.