SGER: Collaborative Research: Contextual Machine Translation

SGER：协作研究：上下文机器翻译尽管机器翻译取得了重大进展，但在解决真实世界上下文中自然多方人类交互的大规模翻译问题上所做的工作很少。多方互动通常发生在这样的背景下，即参与者可以利用多模式事件，如手势、头势、凝视、身体动作等来解释人类语言并建立共同点。这一观察结果导致了这样的假设，即对多通道环境和交互话语进行建模可以提高机器翻译中的自动语言翻译能力。基于这一假设，本探索性研究使用AMI会议语料库的一个子集来研究多通道在多方对话机器翻译中的作用。以用户手势和演示幻灯片为重点确定了适当级别的多模式表示。对指代短语的正确指代和对歧义单词的正确含义进行了注释，并用于评估多模式语境是否以及如何提高指涉解析和词义消除歧义。利用多通道信息的增强语义处理被结合到英德统计机器翻译中。其目的是进行概念验证实验，探索多模式和语篇信息在现实世界中统计机器翻译的有用性。这项探索性研究的结果将为多模式信息的自动提取、使用多模式信息的语言翻译以及将这些信息纳入统计机器翻译的算法和系统提供见解。带注释的数据将提供给研究界，以促进翻译技术的开发，以便在现实世界互动的情况下产生更自然、更像人类的翻译。

SGER: Collaborative Research: Contextual Machine TranslationDespite significant progress in machine translation, little work has been done to address large-scale translation of natural multi-party human interactions in real-world contexts. Multi-party interactions typically take place in a context where multimodal events such as hand gestures, head gestures, gaze, body movements, etc. are available and utilized by participants to interpret human language and establish common ground. This observation leads to the hypothesis that modeling multimodal environment and interaction discourse can improve automated language interpretation for the purpose of machine translation. Based on this hypothesis, this exploratory research investigates the role of multimodality in machine translation of multi-party conversations using a subset of the AMI meeting corpus. An appropriate level of multimodal representation is identified focusing on user gestures and presentation slides. Correct referents to referring expressions and correct senses to ambiguous words are annotated and used to evaluate whether and how multimodal context improves reference resolution and word sense disambiguation. The enhanced semantic processing utilizing multimodal information is incorporated in statistical machine translation for English-German. The objective is to conduct proof-of-concept experiments exploring the usefulness of multimodal and discourse information for statistical machine translation in real-world contexts. The results from this exploratory study will provide insights on algorithms and systems for automatic extraction of multimodal information, language interpretation using multimodal information, and incorporation of this information into statistical machine translation. The annotated data will be made available to the research community to facilitate the development of translation technology that produces more natural, human-like translations in real-world interactive situations.