Copy that! Editing Sequences by Copying Spans
Copy that! Editing Sequences by Copying Spans
复制标题
DOI：
10.1609/aaai.v35i15.17606
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Sheena Panthaplackel;Miltiadis Allamanis;Marc Brockschmidt
中科院分区：
文献类型：
--
作者：
Sheena Panthaplackel;Miltiadis Allamanis;Marc Brockschmidt
神经序列到序列模型越来越多地用于编辑文档，例如纠正文本文档或修复源代码。在本文中，我们认为常见的seq2seq模型（具有复制单个令牌的功能）并不适合这些任务，因为它们必须显式复制每个未更改的令牌。我们提出了一个扩展的seq2seq模型能够复制整个跨度的输入输出在一个步骤中，大大减少了推理过程中所需的决策数量。这种扩展意味着现在有许多方法可以生成相同的输出，我们通过导出一个新的训练目标和一个波束搜索的变体来处理这个问题。在我们对自然语言和源代码的一系列编辑任务的实验中，我们表明我们的新模型始终优于简单的基线。
Neural sequence-to-sequence models are finding increasing use in editing of documents, for example in correcting a text document or repairing source code. In this paper, we argue that common seq2seq models (with a facility to copy single tokens) are not a natural fit for such tasks, as they have to explicitly copy each unchanged token. We present an extension of seq2seq models capable of copying entire spans of the input to the output in one step, greatly reducing the number of decisions required during inference. This extension means that there are now many ways of generating the same output, which we handle by deriving a new objective for training and a variation of beam search for inference that explicitly handles this problem. In our experiments on a range of editing tasks of natural language and source code, we show that our new model consistently outperforms simpler baselines.