Incremental Beam Manipulation for Natural Language Generation
Incremental Beam Manipulation for Natural Language Generation
复制标题
DOI：
10.18653/v1/2021.eacl-main.219
复制
发表时间：
2021-02
期刊：
影响因子：
--
通讯作者：
J. Hargreaves;Andreas Vlachos;Guy Edward Toh Emerson
中科院分区：
文献类型：
--
作者：
J. Hargreaves;Andreas Vlachos;Guy Edward Toh Emerson
随着现代神经网络的发展，自然语言生成系统的性能有了很大的提高。在测试时，他们通常使用波束搜索来避免局部最优但全局次优的预测。然而，根据评估指标，由于模型误差，较大的波束尺寸可能会导致性能恶化。因此，对波束搜索的输出进行重新排序是很常见的，但这依赖于波束搜索来产生一组良好的假设，这限制了潜在的收益。与波束搜索相比，波束搜索的其他替代方案需要改变模型的训练，这限制了它们的适用性。本文提出了增量波束操作，即在解码过程中对波束中的假设进行重新排序，而不仅仅是在解码结束时。这样，不太可能带来良好最终产出的假设就会被抛弃，取而代之的是会被忽视的假设。对于E2E和WebNLG挑战的测试集，采用增量波束操作分别比香草波束搜索提高了1.93点和5.82点。在E2E挑战上，该方法的表现也比强大的重排者高出1.04分，而在WebNLG数据集上则与之持平。
The performance of natural language generation systems has improved substantially with modern neural networks. At test time they typically employ beam search to avoid locally optimal but globally suboptimal predictions. However, due to model errors, a larger beam size can lead to deteriorating performance according to the evaluation metric. For this reason, it is common to rerank the output of beam search, but this relies on beam search to produce a good set of hypotheses, which limits the potential gains. Other alternatives to beam search require changes to the training of the model, which restricts their applicability compared to beam search. This paper proposes incremental beam manipulation, i.e. reranking the hypotheses in the beam during decoding instead of only at the end. This way, hypotheses that are unlikely to lead to a good final output are discarded, and in their place hypotheses that would have been ignored will be considered instead. Applying incremental beam manipulation leads to an improvement of 1.93 and 5.82 BLEU points over vanilla beam search for the test sets of the E2E and WebNLG challenges respectively. The proposed method also outperformed a strong reranker by 1.04 BLEU points on the E2E challenge, while being on par with it on the WebNLG dataset.