EAGER: Collaborative Research: Scaling Up Discriminative Learning for Natural Language Understanding and Translation

点击翻译按钮获取中文摘要

This EArly Grant for Exploratory Research aims to improve automatic understanding of natural language by machines, and automatic translation between languages such as Chinese and English. In the realm of understanding, the project develops methods for syntactically and semantically analyzing, or parsing, sentences. Improved parsing can help in accessing the enormous amount of information available in unstructured text on the web and in databases of newspapers and scanned books. Improved translation between languages increases opportunities for trade as well as for dissemination of information generally between nations and cultures. Machine translation is widely used today despite its generally poor quality, and any improvement in quality will improve access to information for millions of people. This project aims to exploit the power of machine learning algorithms that are designed to discriminate between correct and incorrect outputs by numerically optimizing mathematical functions that are defined in terms of the data available for training. Discriminative structured prediction algorithms have witnessed great success in the field of natural language processing (NLP) over the past decade, generally surpassing their generative counterparts. However, there remain two major problems which prevent discriminative methods from scaling to very large datasets: first, they typically assume exact search (over a prohibitively large search space), which is rarely possible in practice for problems such as parsing and translation. Secondly, they normally assume the data is completely annotated, whereas many naturally occurring datasets are only partially annotated: for example a parallel text in machine translation includes the source and target sentence pairs but not the derivation between them. As a result of these two problems, the current methods are not taking full advantage of the enormous and ever increasing amount of text data available to us.This EArly Grant ofr Exploratory Research (EAGER) aims to: - Develop a linear-time structured learning framework specifically tailored for inexact search, which hopefully retains theoretical properties of structured learning (e.g. convergence) under exact search. - Extend this framework to handle latent variables, such as derivations in machine translation, syntactic structures in semantic parsing, and semantic representations in question answering. If the exploratory extension to latent variable frameworks is sucessful, it will enable longer-term research to: - Apply these efficient learning algorithms to discriminative training of machine translation systems over the entire training dataset rather than only on a small development set. - Apply these efficient learning algorithms to discriminative training for syntactic and semantic parsing, with the goal of scaling up semantic parsing to enable web-scale knowledge extraction.