RI: Collaborative Research: Discriminative Knowledge-Rich Language Modeling for Machine Translation

点击翻译按钮获取中文摘要

This project investigates a novel approach for assessing the fluency andgrammaticality of alternative translation hypotheses that are created withinsearch-based Machine Translation (MT) systems. This task, commonly termed"Language Modeling" (LM), has been explored primarily in the context of speechrecognition; however, current state-of-the-art language models (LMs) are noteffective at distinguishing between more fluent grammatical translations andtheir poor alternatives. In contrast, the proposed approach, "DiscriminativeKnowledge-Rich Language Modeling" (DKRLM), is explicitly designed to find themost fluent and grammatical translations within the search space by comparingthe linguistic features of the translation hypotheses against very large"clean" monolingual corpora. The intuition is that more grammaticaltranslation hypotheses should contain higher proportions of features seen inthe large corpora. An important contribution of the project is in exploringdifferent types of linguistic features to identify those that are mostinformative for the comparisons. Moreover, discriminative training isperformed to incorporate the features into a system-independent scoringfunction, replacing traditional LMs in MT systems. The broader impacts of theproposed work include both broader adoption for the methodology as well aswider use of the new DKRLM functions to other search-based NLP applicationsthat aim at generating fluent grammatical text. This includes search-basedapproaches to Speech Recognition, Natural Language Generation (NLG), OpticalCharacter Recognition (OCR), Summarization, and others.