COMET: A Neural Framework for MT Evaluation
COMET: A Neural Framework for MT Evaluation
复制标题
DOI：
10.18653/v1/2020.emnlp-main.213
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
Ricardo Rei;Craig Alan Stewart;Ana C. Farinha;A. Lavie
中科院分区：
文献类型：
--
作者：
Ricardo Rei;Craig Alan Stewart;Ana C. Farinha;A. Lavie
我们提出了COMET，这是一个训练多语言机器翻译评价模型的神经框架，它与人类判断的相关性达到了新的水平。我们的框架利用了最近在跨语言预训练语言建模方面的突破，从而产生了高度多语言和适应性的机器翻译评估模型，该模型利用来自源输入和目标语言参考翻译的信息，以便更准确地预测机器翻译质量。为了展示我们的框架，我们训练了三个具有不同类型人类判断的模型：直接评估、人工中介的翻译编辑率和多维质量度量。我们的模型在WMT 2019指标共享任务上实现了最新的最先进性能，并展示了对高性能系统的稳健性。
We present COMET, a neural framework for training multilingual machine translation evaluation models which obtains new state-of-the-art levels of correlation with human judgements. Our framework leverages recent breakthroughs in cross-lingual pretrained language modeling resulting in highly multilingual and adaptable MT evaluation models that exploit information from both the source input and a target-language reference translation in order to more accurately predict MT quality. To showcase our framework, we train three models with different types of human judgements: Direct Assessments, Human-mediated Translation Edit Rate and Multidimensional Quality Metrics. Our models achieve new state-of-the-art performance on the WMT 2019 Metrics shared task and demonstrate robustness to high-performing systems.