SGER: Self-Supervised Discriminative Training of Statistical Language Models

标题：统计语言模型的自监督判别训练这项探索性研究的小额赠款正在研究统计语言模型判别训练的新方法，以应用于各种人类语言技术，如自动语音识别（ASR）和机器翻译（MT）。语言模型（LM）通常是通过正则化最大似然从目标域的大量文本语料库中估计出来的。鉴别标准在ASR中的应用取得了一些成功，但由于需要额外的转录语音语料库来区分正确的单词序列和不正确的单词序列，它们的巨大前景受到了限制。该项目正在探索在不需要大量人工注释的情况下判别估计语言模型的方法，即，ASR的转录语音或mt的并行文本。正在探索的关键思想是，如果有大量（例如）单语中文文本可用，然后，通过尝试使用现有的机器翻译系统将这篇文章翻译成英语，并检查哪些英语单词和短语最频繁地相互竞争，就可以准确地估计中文单词和短语的机器翻译队列。在任何特定情况下，没有必要知道队列集合中哪个竞争词或短语是正确的翻译！只要知道谁最常参与竞争就足够了。研究人员使用单语英语文本来探索在观察到的队列组中每个成员的发病率与其假定的竞争对手之间的区别特征；因此，判别训练的数据是综合导出的。他们正在研究这样一个经过鉴别训练的机器学习是否专门针对机器翻译系统所面临的最严重的模糊性。ASR的对应物，即从自动转录无注释语音中衍生的群组集，也正在探索中。该项目通过探索统计语言模型使ASR和MT研究社区受益，这些模型可以在没有人为干预的情况下适应不断变化的任务或语言使用，并且较少依赖于手动注释的数据。ASR和MT的进步反过来将促进更有效的计算机辅助获取多种语言和媒体的信息。

Title: Self-Supervised Discriminative Training of Statistical Language ModelsThis Small Grant for Exploratory Research is investigating novel methods for discriminative training of statistical language models for application to various human language technologies, such as automatic speech recognition (ASR) and machine translation (MT).A language model (LM) is conventionally estimated from a large corpus of text in the target domain via regularized maximum likelihood. Discriminative criteria have been used with some success in ASR, but their immense promise has been curtailed by the requirement of an additional corpus of transcribed speech needed to discriminate between correct word sequences and their incorrect ?cohorts.? This project is exploring ways to discriminatively estimate language models without requiring massive manual annotation, namely, transcribed speech for ASR or parallel text for MT.The key idea being explored is that if a large amount of (say) monolingual Chinese text is available, then the MT cohorts of Chinese words and phrases may be accurately estimated by attempting to translate this text into (say) English using an existing MT system and examining which English words and phrases are most frequently in competition with each other. It is not necessary to know which of the competing words or phrases in a cohort set is the correct translation in any particular instance! It suffices to learn who are most often in competition. The investigators are using monolingual English text to explore features that discriminate between observed incidences of each member of a cohort set and its putative competitors; the data for discriminative training are thus derived synthetically. They are investigating if such a discriminatively trained LM specifically targets the most debilitating ambiguities faced by the MT system. The ASR counterpart, with cohort sets derived from automatic transcription of unannotated speech, is also being explored.This project benefits both the ASR and MT research communities by exploring statistical language models that can adapt without human intervention to changing tasks or language-use, and that are less reliant on manually annotated data. Advances in ASR and MT in turn will facilitate more effective computer-aided access to information in multiple languages and media.