SGER: RI: Text-Based Discriminative Language Modeling

区分训练的条件模型已经成功地应用于语音识别和机器翻译等语言建模问题，并被证明始终优于产生式建模方法。不幸的是，与生成性方法相比，区别性建模是一种完全有监督的方法，需要昂贵的人工注释的训练数据。然而，语言建模和其他有区别地建模的自然语言处理(NLP)任务之间有一个重要的区别。对于其他NLP任务，单词序列是输入，一些隐藏结构是输出，例如，语法分析树；但对于语言建模任务，单词序列是给定某些输入的输出，例如正在翻译的源语言串。然而，没有与感兴趣的输入配对的大型文本资源提供了格式良好的输出的例子，这对于产生它的任何输入来说都是“正确的”。这一探索性建议的新颖视角是从语言建模的角度认识到典型自然语言处理任务之间的根本区别，在典型的自然语言处理任务中，可能有充足的输入，但输出必须手动注释，而在语言建模中，有大量的输出，而没有相应的输入。该项目探索了为观察到的单词序列模拟输入的方法，并将这些模拟输入与特定的机器翻译系统一起使用，以产生原始观察序列的一组替代(易混淆的)单词序列。这些可选序列集合然后可以与条件/区分估计技术一起使用，尽管不需要监督(源串的人工翻译)来产生训练数据。

Discriminatively trained conditional models have been applied with great success to language modeling problems for applications such as speech recognition and machine translation (MT), and have been demonstrated to consistently outperform generative modeling approaches. Unfortunately, in contrast to generative approaches, discriminative modeling is an exclusively supervised approach, requiring costly manually annotated training data. Yet there is an important difference between language modeling and other natural language processing (NLP) tasks that are modeled discriminatively. For other NLP tasks, sequences of words are the input, and some hidden structure is the output, e.g., parse trees; but for language modeling tasks, word sequences are the output given some input, such as a source language string being translated. Large text resources that are not paired with an input of interest nevertheless provide examples of well-formed outputs, which would be 'correct' for any inputs that producing it. The novel perspective in this exploratory proposal is recognizing this fundamental difference between typical NLP tasks, where there may be ample inputs but outputs must be manually annotated, from language modeling, where there are ample outputs with no corresponding input. This project explores methods for simulating inputs for observed word sequences, and for using these simulated inputs with a particular MT system to produce a set of alternative (confusable) word sequences to the original observed sequence. These sets of alternative sequences can then be used with conditional/discriminative estimation techniques, despite the fact that no supervision (manual translation of source strings) was required to produce the training data.