Humans in the Loop: An investigation into the impact of the quality of source texts on the work of post-editors and translators at TranslateMedia

TranslateMedia是一家领先的翻译公司，专门从事广告和营销材料的翻译。该公司正在实施自适应神经机器翻译（ANMT）的后期编辑，即人工校正机器翻译（MT）输出，作为其专业工作流程的一部分。非规范的和写得不好的源文本是机器翻译错误的巨大来源，因为有缺陷的输入通常会产生有缺陷的输出（盖革等人，2019年），这是将机器翻译有效整合到翻译工作流程中的一个重大挑战。本研究将考察源文本质量对机器翻译输出、译后工作量、译者生产力和专业实践的影响。神经机器翻译（NMT）的译后工作量研究仍然很少（Jia et al.，2019年），以前没有研究集中在其用于广告和营销材料的翻译，也没有对源文本的质量对MT输出和后期编辑工作的影响。此外，将机器翻译纳入专业工作流程涉及翻译实践的重大变化，因此，人力和组织方面对新流程是否成功起着关键作用（Olohan，2011）。然而，翻译研究对翻译技术的社会层面及其与翻译实践的互动关注相对较少（Olohan，2011; 2017; 2020）。源文本的质量如何影响TranslateMedia的后期编辑和翻译人员的工作，使用自适应神经机器翻译和全人工后期编辑来翻译广告和营销材料？- 写得不好和不规范的源文本在多大程度上影响机器翻译输出的质量？- 原文质量如何影响后期编辑实践？- 写得不好和不规范的源文本对后期编辑的工作效率和质量有什么影响？- 低质量和不标准的源文本如何影响估计的、真实的和感知的编辑后努力之间的相关性？回答这些问题将对我们理解机器翻译技术的最新发展如何改变专业翻译工作流程和实践做出重大贡献。本项目旨在阐明的一些最有趣的方面包括：（a）低质量源文本如何影响语言学家的生产力，（B）当翻译后编辑来自低质量源文本的机器翻译输出时，翻译人员是否改变了他们的专业实践，以及（c）在这些情况下，译者对后期编辑工作的看法是否符合真实的后期编辑工作，这项研究将调查英语源文本的质量对翻译成TranslateMedia最受欢迎的欧洲目标语言（即法语，德语，意大利语，西班牙语，葡萄牙语和荷兰语）的MT输出的影响。TranslateMedia将提供对数据集和资源的访问，这些数据集和资源将用于创建一个非规范和书写不佳的源文本语料库，以及一个规模和语言领域相当但语言标准更高的语料库。两个语料库的机器翻译质量的自动评估将与人工机器翻译质量评估的结果进行比较，人工机器翻译质量评估将由TranslateMedia的自由译员进行。将使用问卷和半结构化访谈来查明机器翻译输入的质量是否影响编辑后的做法，以及对笔译员的生产率有何影响。例如，这些问题将包括翻译产出的质量是否影响笔译员对编辑后工作的态度、他们对工作质量和专业作用的满意度。还将包括关于感觉到的编辑后工作量的问题，并将其与自动估计和实际（衡量的）编辑后工作量进行比较。

TranslateMedia is a leading translation company that specialises in the translation of advertising and marketing materials. The company is implementing post-editing of adaptive neural machine translation (ANMT), i.e. human correction of machine translation (MT) output, as part of its professional workflow. Non-canonical and poorly written source texts are a huge source of errors for MT, because flawed input will normally generate flawed output (Geiger et al., 2019), constituting a major challenge for the effective integration of MT into the translation workflow. This research will examine the impact of the quality of source texts on MT output, post-editing effort, translators' productivity and professional practices.Research on post-editing of neural machine translation (NMT) is still scarce (Jia et al., 2019) and no previous research has focused on its use for the translation of advertising and marketing materials, nor on the impact of the quality of source texts on MT output and on the work of post-editors. Moreover, the integration of MT into the professional workflow involves a significant change in translation practices, therefore human and organisational aspects play a key role on whether the new process will be successful (Olohan, 2011). However, Translation Studies research has paid relatively little attention to the social aspect of translation technology and its interaction with translation practices (Olohan, 2011; 2017; 2020).How does the quality of source texts affect the work of post-editors and translators at TranslateMedia, using adaptive neural machine translation and full human post-editing to translate advertising and marketing materials? - To what extent do poorly written and non-canonical source texts affect the quality of MT output? - How does the quality of source texts affect post-editing practices? - What impact do poorly written and non-canonical source texts have on the productivity and quality of work of post-editors? - How do low-quality and non-standard source texts affect the correlation between estimated, real and perceived post-editing effort? Answering these questions will make a significant contribution to our understanding of how latest developments in MT technology are changing professional translation workflows and practices. Some of the most interesting aspects that this project will aim to shed light on include (a) how low-quality source texts affect linguists' productivity, (b) whether translators change their professional practices when post-editing MT output from low-quality source texts, and (c) whether translators' perceptions of post-editing effort under these circumstances correspond to the real post-editing effort.This research will investigate the impact of the quality of English source texts on the MT output into TranslateMedia's most requested European target languages, i.e. French, German, Italian, Spanish, Portuguese and Dutch. TranslateMedia will provide access to datasets and resources, that will be used to create a corpus of non-canonical and poorly written source texts, and a corpus of comparable size and linguistic domain, but more linguistically standard. Automatic estimations of MT quality for both corpora will be compared to results from human MT quality assessment, which will be undertaken by TranslateMedia's freelance translators. Questionnaires and semi-structured interviews will be used to find out whether the quality of MT input affects post-editing practices and what are the implications on translators' productivity. For example, they will include questions on whether the quality of the MT output affects translators' attitude towards the post-editing task, their satisfaction with the quality of their work and with their professional role. They will also include questions on perceived post-editing effort, which will be compared to automatic estimations and the actual (measured) post-editing effort.