Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation
复制标题
DOI：
10.18653/v1/2021.acl-long.562
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Eleftheria Briakou;Marine Carpuat
中科院分区：
文献类型：
--
作者：
Eleftheria Briakou;Marine Carpuat
虽然已经证明神经机器翻译（NMT）对有噪声的并行训练样本高度敏感，但以前的工作将源和目标之间的所有类型的不匹配都视为噪声。因此，目前尚不清楚大多数情况下等价但包含少量语义上不同的标记的样本如何影响NMT训练。为了缩小这一差距，我们分析了不同类型的细粒度语义分歧对Transformer模型的影响。我们发现，在合成分歧上训练的模型更频繁地输出退化的文本，并且对它们的预测不太有信心。基于这些发现，我们引入了一个发散感知的NMT框架，该框架使用因子来帮助NMT从自然发生的发散引起的退化中恢复，从而提高EN-FR任务的翻译质量和模型校准。
While it has been shown that Neural Machine Translation (NMT) is highly sensitive to noisy parallel training samples, prior work treats all types of mismatches between source and target as noise. As a result, it remains unclear how samples that are mostly equivalent but contain a small number of semantically divergent tokens impact NMT training. To close this gap, we analyze the impact of different types of fine-grained semantic divergences on Transformer models. We show that models trained on synthetic divergences output degenerated text more frequently and are less confident in their predictions. Based on these findings, we introduce a divergent-aware NMT framework that uses factors to help NMT recover from the degradation caused by naturally occurring divergences, improving both translation quality and model calibration on EN-FR tasks.