Humans in the Loop: An investigation into the impact of the quality of source texts on the work of post-editors and translators at TranslateMedia

TranslateMedia是一家领先的翻译公司，专门从事广告和营销材料的翻译。该公司正在实施自适应神经机器翻译（ANMT）的后期编辑，即人工校正机器翻译（MT）输出，作为其专业工作流程的一部分。非规范和写得不好的源文本是机器翻译的巨大错误来源，因为有缺陷的输入通常会产生有缺陷的输出（Geiger等人，2019），这是将机器翻译有效集成到翻译工作流程中的主要挑战。本研究将考察源文本的质量对机器翻译输出、后期编辑工作、译者的生产力和专业实践的影响。关于神经机器翻译（NMT）后编辑的研究仍然很少（Jia et al., 2019），之前也没有研究将其用于广告和营销材料的翻译，也没有研究源文本质量对机器翻译输出和后编辑工作的影响。此外，将机器翻译整合到专业工作流程中涉及翻译实践的重大变化，因此人员和组织方面对新流程是否成功起着关键作用（Olohan, 2011）。然而，翻译研究对翻译技术的社会层面及其与翻译实践的互动关注相对较少（Olohan, 2011; 2017; 2020）。TranslateMedia使用自适应神经机器翻译和全人工后期编辑来翻译广告和营销材料，源文本的质量如何影响后期编辑和翻译人员的工作？写得不好和不规范的源文本在多大程度上影响机器翻译输出的质量？-源文本的质量如何影响后期编辑实践？-写得不好和不规范的源文本对后期编辑的工作效率和质量有什么影响？-低质量和非标准源文本如何影响预估、实际和感知的后期编辑工作量之间的相关性？回答这些问题将对我们理解机器翻译技术的最新发展如何改变专业翻译工作流程和实践做出重大贡献。本项目旨在揭示的一些最有趣的方面包括：(a)低质量的源文本如何影响语言学家的生产力，(b)翻译人员在对低质量源文本的MT输出进行后期编辑时是否会改变他们的专业实践，以及(c)在这种情况下，翻译人员对后期编辑工作的看法是否与实际的后期编辑工作相对应。本研究将调查英语源文本的质量对翻译成TranslateMedia最需要的欧洲目标语言（即法语、德语、意大利语、西班牙语、葡萄牙语和荷兰语）的MT输出的影响。TranslateMedia将提供对数据集和资源的访问，这些数据集和资源将被用来创建一个非规范和写得不好的源文本的语料库，以及一个大小和语言领域相当的语料库，但在语言上更标准。这两个语料库的机器翻译质量的自动估计将与人工机器翻译质量评估的结果进行比较，这将由TranslateMedia的自由译者承担。问卷调查和半结构化访谈将用于了解翻译输入的质量是否影响后期编辑实践，以及对译者生产力的影响。例如，他们将包括MT输出的质量是否影响译者对后期编辑任务的态度，他们对工作质量的满意度和他们的职业角色的问题。它们还将包括关于感知到的后期编辑工作的问题，这将与自动估计和实际的（测量的）后期编辑工作进行比较。

TranslateMedia is a leading translation company that specialises in the translation of advertising and marketing materials. The company is implementing post-editing of adaptive neural machine translation (ANMT), i.e. human correction of machine translation (MT) output, as part of its professional workflow. Non-canonical and poorly written source texts are a huge source of errors for MT, because flawed input will normally generate flawed output (Geiger et al., 2019), constituting a major challenge for the effective integration of MT into the translation workflow. This research will examine the impact of the quality of source texts on MT output, post-editing effort, translators' productivity and professional practices.Research on post-editing of neural machine translation (NMT) is still scarce (Jia et al., 2019) and no previous research has focused on its use for the translation of advertising and marketing materials, nor on the impact of the quality of source texts on MT output and on the work of post-editors. Moreover, the integration of MT into the professional workflow involves a significant change in translation practices, therefore human and organisational aspects play a key role on whether the new process will be successful (Olohan, 2011). However, Translation Studies research has paid relatively little attention to the social aspect of translation technology and its interaction with translation practices (Olohan, 2011; 2017; 2020).How does the quality of source texts affect the work of post-editors and translators at TranslateMedia, using adaptive neural machine translation and full human post-editing to translate advertising and marketing materials? - To what extent do poorly written and non-canonical source texts affect the quality of MT output? - How does the quality of source texts affect post-editing practices? - What impact do poorly written and non-canonical source texts have on the productivity and quality of work of post-editors? - How do low-quality and non-standard source texts affect the correlation between estimated, real and perceived post-editing effort? Answering these questions will make a significant contribution to our understanding of how latest developments in MT technology are changing professional translation workflows and practices. Some of the most interesting aspects that this project will aim to shed light on include (a) how low-quality source texts affect linguists' productivity, (b) whether translators change their professional practices when post-editing MT output from low-quality source texts, and (c) whether translators' perceptions of post-editing effort under these circumstances correspond to the real post-editing effort.This research will investigate the impact of the quality of English source texts on the MT output into TranslateMedia's most requested European target languages, i.e. French, German, Italian, Spanish, Portuguese and Dutch. TranslateMedia will provide access to datasets and resources, that will be used to create a corpus of non-canonical and poorly written source texts, and a corpus of comparable size and linguistic domain, but more linguistically standard. Automatic estimations of MT quality for both corpora will be compared to results from human MT quality assessment, which will be undertaken by TranslateMedia's freelance translators. Questionnaires and semi-structured interviews will be used to find out whether the quality of MT input affects post-editing practices and what are the implications on translators' productivity. For example, they will include questions on whether the quality of the MT output affects translators' attitude towards the post-editing task, their satisfaction with the quality of their work and with their professional role. They will also include questions on perceived post-editing effort, which will be compared to automatic estimations and the actual (measured) post-editing effort.