RI: Medium: Collaborative Research: Semi-Supervised Discriminative Training of Language Models

RI：媒介：协作研究：语言模型的半监督判别训练

• 批准号: 0964102
• 负责人: Alexander Kain
• 金额: $ 50万
• 依托单位: Oregon Health & Science University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-06-01 至 2015-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0964102&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Semi

This project is conducting fundamental research in statistical language modeling to improve human language technologies, including automatic speech recognition (ASR) and machine translation (MT).A language model (LM) is conventionally optimized, using text in the target language, to assign high probability to well-formed sentences. This method has a fundamental shortcoming: the optimization does not explicitly target the kinds of distinctions necessary to accomplish the task at hand, such as discriminating (for ASR) between different words that are acoustically confusable or (for MT) between different target-language words that express the multiple meanings of a polysemous source-language word.Discriminative optimization of the LM, which would overcome this shortcoming, requires large quantities of paired input-output sequences: speech and its reference transcription for ASR or source-language (e.g. Chinese) sentences and their translations into the target language (say, English) for MT. Such resources are expensive, and limit the efficacy of discriminative training methods.In a radical departure from convention, this project is investigating discriminative training using easily available, *unpaired* input and output sequences: un-transcribed speech or monolingual source-language text and unpaired target-language text. Two key ideas are being pursued: (i) unlabeled input sequences (e.g. speech or Chinese text) are processed to learn likely confusions encountered by the ASR or MT system; (ii) unpaired output sequences (English text) are leveraged to discriminate between these well-formed sentences from the (supposed) ill-formed sentences the system could potentially confuse them with.This self-supervised discriminative training, if successful, will advance machine intelligence in fundamental ways that impact many other applications.

本研究课题为提高自动语音识别（ASR）和机器翻译（MT）等人类语言技术，进行统计语言建模的基础研究。使用目标语言的文本，对语言模型（LM）进行常规优化，使正确的句子具有较高的概率。 这种方法有一个根本的缺点：优化没有明确地针对完成手头任务所需的各种区别，例如区分（对于ASR）在听觉上易混淆的不同单词之间或者（对于MT）在表达多义源语言单词的多种含义的不同目标语言单词之间。LM的区分性优化，其将克服这个缺点，需要大量成对的输入-输出序列：语音及其用于ASR的参考转录或源语言（例如中文）句子及其到用于MT的目标语言（例如英语）的翻译。 这种资源是昂贵的，并限制了判别训练方法的有效性。在一个彻底的背离惯例，这个项目正在研究判别训练使用容易获得的，* 不成对的 * 输入和输出序列：未转录的语音或单语源语言文本和不成对的目标语言文本。 两个关键的想法正在追求：（i）未标记的输入序列（例如语音或中文文本）被处理以学习ASR或MT系统可能遇到的混淆;（ii）不成对输出序列（英文文本）被用来区分这些格式良好的句子，这种自我监督的区别训练，如果成功，将以影响许多其他应用的根本方式推进机器智能。