EAGER: Collaborative Research: Scaling Up Discriminative Learning for Natural Language Understanding and Translation

EAGER：协作研究：扩大自然语言理解和翻译的判别学习

• 批准号: 1656051
• 负责人: Liang Huang
• 金额: $ 9.04万
• 依托单位: Oregon State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-08-27 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1656051&HistoricalAwards=false
• 关键词: EAGER; Collaborative; Research; Scaling; Up

This EArly Grant for Exploratory Research aims to improve automatic understanding of natural language by machines, and automatic translation between languages such as Chinese and English. In the realm of understanding, the project develops methods for syntactically and semantically analyzing, or parsing, sentences. Improved parsing can help in accessing the enormous amount of information available in unstructured text on the web and in databases of newspapers and scanned books. Improved translation between languages increases opportunities for trade as well as for dissemination of information generally between nations and cultures. Machine translation is widely used today despite its generally poor quality, and any improvement in quality will improve access to information for millions of people. This project aims to exploit the power of machine learning algorithms that are designed to discriminate between correct and incorrect outputs by numerically optimizing mathematical functions that are defined in terms of the data available for training. Discriminative structured prediction algorithms have witnessed great success in the field of natural language processing (NLP) over the past decade, generally surpassing their generative counterparts. However, there remain two major problems which prevent discriminative methods from scaling to very large datasets: first, they typically assume exact search (over a prohibitively large search space), which is rarely possible in practice for problems such as parsing and translation. Secondly, they normally assume the data is completely annotated, whereas many naturally occurring datasets are only partially annotated: for example a parallel text in machine translation includes the source and target sentence pairs but not the derivation between them. As a result of these two problems, the current methods are not taking full advantage of the enormous and ever increasing amount of text data available to us.This EArly Grant ofr Exploratory Research (EAGER) aims to: - Develop a linear-time structured learning framework specifically tailored for inexact search, which hopefully retains theoretical properties of structured learning (e.g. convergence) under exact search. - Extend this framework to handle latent variables, such as derivations in machine translation, syntactic structures in semantic parsing, and semantic representations in question answering. If the exploratory extension to latent variable frameworks is sucessful, it will enable longer-term research to: - Apply these efficient learning algorithms to discriminative training of machine translation systems over the entire training dataset rather than only on a small development set. - Apply these efficient learning algorithms to discriminative training for syntactic and semantic parsing, with the goal of scaling up semantic parsing to enable web-scale knowledge extraction.

EArly探索性研究基金旨在提高机器对自然语言的自动理解，以及中文和英文等语言之间的自动翻译。在理解领域，该项目开发了句法和语义分析或解析句子的方法。改进的解析可以帮助访问网络上的非结构化文本以及报纸和扫描书籍数据库中的大量信息。语言间翻译的改进增加了各国和各文化间贸易和信息传播的机会。尽管机器翻译的质量普遍较差，但它在今天仍被广泛使用，质量的任何改进都将改善数百万人对信息的获取。 该项目旨在利用机器学习算法的能力，该算法旨在通过数值优化根据可用于训练的数据定义的数学函数来区分正确和不正确的输出。 在过去的十年中，判别式结构化预测算法在自然语言处理（NLP）领域取得了巨大的成功，通常超过了生成式算法。然而，仍然存在两个主要问题，阻止区分方法扩展到非常大的数据集：首先，它们通常假设精确搜索（在非常大的搜索空间上），这在实际中很少可能用于解析和翻译等问题。其次，它们通常假设数据是完全注释的，而许多自然发生的数据集只被部分注释：例如，机器翻译中的平行文本包括源和目标句子对，但不包括它们之间的派生。由于这两个问题，目前的方法并没有充分利用巨大的和不断增加的文本数据提供给我们。EARLY授予ofr探索性研究（EAGER）的目的是：-开发一个线性时间结构化学习框架，专门为不精确搜索量身定制，希望保留精确搜索下的结构化学习的理论属性（例如收敛）。 - 扩展此框架以处理潜在变量，例如机器翻译中的派生、语义分析中的句法结构和问答中的语义表示。 如果对潜在变量框架的探索性扩展成功，它将使长期研究能够：-将这些高效的学习算法应用于整个训练数据集而不是仅在小的开发集上对机器翻译系统进行区分训练。 - 将这些高效的学习算法应用于句法和语义解析的区分训练，目标是扩展语义解析，以实现Web规模的知识提取。

项目成果

Learning to Stop in Structured Prediction for Neural Machine Translation

• DOI: 10.18653/v1/n19-1187
• 发表时间: 2019-04
• 作者: Mingbo Ma;Renjie Zheng;Liang Huang

Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report

• DOI: 10.18653/v1/w18-6443
• 发表时间: 2018-08
• 作者: Renjie Zheng;Yilin Yang;Mingbo Ma;Liang Huang

Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation

• DOI: 10.18653/v1/d18-1357
• 发表时间: 2018-08
• 期刊: ArXiv
• 作者: Renjie Zheng;Mingbo Ma;Liang Huang

Speeding Up Neural Machine Translation Decoding by Cube Pruning

• DOI: 10.18653/v1/d18-1460
• 发表时间: 2018-09
• 期刊: ArXiv
• 作者: Wen Zhang;Liang Huang;Yang Feng;Lei Shen;Qun Liu

Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

• DOI: 10.18653/v1/d18-1342
• 发表时间: 2018-08
• 期刊: ArXiv
• 作者: Yilin Yang;Liang Huang;Mingbo Ma