SGER: Collaborative Research: Contextual Machine Translation

SGER：合作研究：上下文机器翻译尽管机器翻译取得了重大进展，但在现实世界背景下解决自然多方人类交互的大规模翻译方面却做得很少。多方交互通常发生在多模态事件（诸如手势、头部姿势、凝视、身体运动等）可用并且被参与者用来解释人类语言并建立共同点的上下文中。这一观察导致了这样一个假设，即建模多模态环境和交互话语可以提高机器翻译的自动语言解释。 基于这一假设，本研究利用AMI会议语料库的一个子集，探讨了多模态在多方对话机器翻译中的作用。 一个适当的多模态表示的水平被确定为专注于用户手势和演示幻灯片。引用表达式的正确引用和歧义词的正确含义被注释并用于评估多模态上下文是否以及如何提高引用分辨率和词义消歧。利用多模态信息的增强语义处理被纳入英德统计机器翻译中。 我们的目标是进行概念验证实验，探索多模态和话语信息在现实世界中的统计机器翻译的有用性。 这项探索性研究的结果将为多模态信息的自动提取、使用多模态信息的语言解释以及将这些信息纳入统计机器翻译的算法和系统提供见解。注释的数据将提供给研究界，以促进翻译技术的发展，在现实世界的互动情况下产生更自然，更人性化的翻译。

SGER: Collaborative Research: Contextual Machine TranslationDespite significant progress in machine translation, little work has been done to address large-scale translation of natural multi-party human interactions in real-world contexts. Multi-party interactions typically take place in a context where multimodal events such as hand gestures, head gestures, gaze, body movements, etc. are available and utilized by participants to interpret human language and establish common ground. This observation leads to the hypothesis that modeling multimodal environment and interaction discourse can improve automated language interpretation for the purpose of machine translation. Based on this hypothesis, this exploratory research investigates the role of multimodality in machine translation of multi-party conversations using a subset of the AMI meeting corpus. An appropriate level of multimodal representation is identified focusing on user gestures and presentation slides. Correct referents to referring expressions and correct senses to ambiguous words are annotated and used to evaluate whether and how multimodal context improves reference resolution and word sense disambiguation. The enhanced semantic processing utilizing multimodal information is incorporated in statistical machine translation for English-German. The objective is to conduct proof-of-concept experiments exploring the usefulness of multimodal and discourse information for statistical machine translation in real-world contexts. The results from this exploratory study will provide insights on algorithms and systems for automatic extraction of multimodal information, language interpretation using multimodal information, and incorporation of this information into statistical machine translation. The annotated data will be made available to the research community to facilitate the development of translation technology that produces more natural, human-like translations in real-world interactive situations.