Human Evaluation of Grammatical Error Correction Systems
Human Evaluation of Grammatical Error Correction Systems
复制标题
语法纠错系统的人工评估
DOI：
--
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
Edward Gillian
中科院分区：
文献类型：
--
作者：
Roman Grundkiewicz;Marcin Junczys;Edward Gillian
该论文介绍了自动语法错误纠正（GEC）系统的首次大规模人类评估的结果。十二个参与系统和 CoNLL-2014 共享任务的不变输入已在受 WMT 启发的人类评估程序中进行了重新评估。机器翻译研讨会评估活动引入的方法已适应 GEC 并在必要时进行扩展。生成的排名用于根据与人类判断的相关性来评估语法错误纠正的标准指标。
The paper presents the results of the first large-scale human evaluation of automatic grammatical error correction (GEC) systems. Twelve participating systems and the unchanged input of the CoNLL-2014 shared task have been reassessed in a WMT-inspired human evaluation procedure. Methods introduced for the Workshop of Machine Translation evaluation campaigns have been adapted to GEC and extended where necessary. The produced rankings are used to evaluate standard metrics for grammatical error correction in terms of correlation with human judgment.