Measuring Translation Quality by Testing English Speakers with a New Defense Language Proficiency Test for Arabic
Measuring Translation Quality by Testing English Speakers with a New Defense Language Proficiency Test for Arabic
复制标题
通过新的阿拉伯语国防语言能力测试测试英语使用者来衡量翻译质量
DOI：
--
复制
发表时间：
2005
期刊：
影响因子：
--
通讯作者：
C. Weinstein
中科院分区：
文献类型：
--
作者：
Douglas A. Jones;Wade Shen;Neil Granoien;M. Herzog;C. Weinstein
摘要：我们展示了一项实验的结果，在该实验中，受过教育的英语母语人士回答了标准化阿拉伯语测试的机器翻译版本中的问题。我们将机器翻译 (MT) 结果与专业参考翻译作为基准进行比较，以确定当前机器翻译技术使英语使用者能够达到的阿拉伯语阅读理解水平。此外，我们还探讨了当前被广泛接受的机器翻译自动性能衡量标准与国防语言能力测试（一种被广泛接受的评估外语能力有效性的衡量标准）之间的关系。在此过程中，我们打算帮助将机器翻译系统的性能转化为对满足政府外语处理要求有意义的术语。该实验的结果表明，机器翻译可以实现机构间语言圆桌会议 2 级性能，但还不足以达到 ILR 3 级。我们的结果基于 69 名受试者阅读 68 份文档并回答 173 个问题，总共进行了 4,692 个定时文档试验和 7,950 个问题试验。我们建议将 Level 3 作为机器翻译研究和开发的合理近期目标。
Abstract : We present results from an experiment in which educated English-native speakers answered questions from a machine translated version of a standardized Arabic language test. We compare the machine translation (MT) results with professional reference translations as a baseline for the purpose of determining the level of Arabic reading comprehension that current machine translation technology enables an English speaker to achieve. Furthermore, we explore the relationship between the current, broadly accepted automatic measures of performance for machine translation and the Defense Language Proficiency Test, a broadly accepted measure of effectiveness for evaluating foreign language proficiency. In doing so, we intend to help translate MT system performance into terms that are meaningful for satisfying Government foreign language processing requirements. The results of this experiment suggest that machine translation may enable Interagency Language Roundtable Level 2 performance, but is not yet adequate to achieve ILR Level 3. Our results are based on 69 human subjects reading 68 documents and answering 173 questions, giving a total of 4,692 timed document trials and 7,950 question trials. We propose Level 3 as a reasonable near-term target for machine translation research and development.