SGER: Self-Supervised Discriminative Training of Statistical Language Models

SGER：统计语言模型的自监督判别训练

• 批准号: 0840112
• 负责人: Sanjeev Khudanpur
• 金额: --
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-09-01 至 2010-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0840112&HistoricalAwards=false
• 关键词: SGER; Self; Supervised; Discriminative; Training

Title: Self-Supervised Discriminative Training of Statistical Language ModelsThis Small Grant for Exploratory Research is investigating novel methods for discriminative training of statistical language models for application to various human language technologies, such as automatic speech recognition (ASR) and machine translation (MT).A language model (LM) is conventionally estimated from a large corpus of text in the target domain via regularized maximum likelihood. Discriminative criteria have been used with some success in ASR, but their immense promise has been curtailed by the requirement of an additional corpus of transcribed speech needed to discriminate between correct word sequences and their incorrect ?cohorts.? This project is exploring ways to discriminatively estimate language models without requiring massive manual annotation, namely, transcribed speech for ASR or parallel text for MT.The key idea being explored is that if a large amount of (say) monolingual Chinese text is available, then the MT cohorts of Chinese words and phrases may be accurately estimated by attempting to translate this text into (say) English using an existing MT system and examining which English words and phrases are most frequently in competition with each other. It is not necessary to know which of the competing words or phrases in a cohort set is the correct translation in any particular instance! It suffices to learn who are most often in competition. The investigators are using monolingual English text to explore features that discriminate between observed incidences of each member of a cohort set and its putative competitors; the data for discriminative training are thus derived synthetically. They are investigating if such a discriminatively trained LM specifically targets the most debilitating ambiguities faced by the MT system. The ASR counterpart, with cohort sets derived from automatic transcription of unannotated speech, is also being explored.This project benefits both the ASR and MT research communities by exploring statistical language models that can adapt without human intervention to changing tasks or language-use, and that are less reliant on manually annotated data. Advances in ASR and MT in turn will facilitate more effective computer-aided access to information in multiple languages and media.

职务名称：统计语言模型的自我监督判别训练这个探索性研究的小额资助正在研究统计语言模型的判别训练的新方法，以应用于各种人类语言技术，如自动语音识别（ASR）和机器翻译（MT）。语言模型（LM）通常通过正则化最大似然法从目标域的大型文本语料库中估计。 判别标准已被用于与一些成功的ASR，但他们的巨大的承诺已削减的要求额外语料库的转录语音需要区分正确的单词序列和他们的不正确的？同伙？ 该项目正在探索在不需要大量人工注释的情况下有区别地估计语言模型的方法，即ASR的转录语音或MT的平行文本。正在探索的关键思想是，如果有大量（比如）单语中文文本可用，则通过尝试将该文本翻译成（例如）英语使用现有的机器翻译系统，并检查哪些英语单词和短语是最经常相互竞争的。 在任何特定情况下，不需要知道群组集中的哪个竞争词或短语是正确的翻译！ 只要知道谁最常参加竞争就够了。 研究人员正在使用单语英语文本来探索区分队列集合中每个成员及其假定竞争对手的观察到的发病率的特征;因此，用于区分训练的数据是综合得出的。 他们正在研究这种有区别地训练的LM是否专门针对MT系统所面临的最令人衰弱的模糊性。 ASR对应的队列集来自自动转录的未注释的语音，也正在探索中。该项目通过探索统计语言模型使ASR和MT研究社区受益，这些模型可以在没有人为干预的情况下适应不断变化的任务或语言使用，并且较少依赖手动注释的数据。 自动检索和机器翻译方面的进展反过来又将促进更有效地利用计算机辅助获取多种语言和媒体的信息。