RI: Parsing Models and Algorithms for Morphologically Rich Languages

到目前为止，自然语言处理（NLP）领域主要将精力集中在英语技术上，尽管英语是一个典型的异常值，而且世界上大多数人都不会说英语。本项目旨在开发统计自然语言分析工具来消除非英语文本的形态和句法结构歧义。具体来说，试点研究的目标是设计、训练、实施和传播阿拉伯语和希伯来语的统计形态句法分析模型。这个项目从一个简单的形式化（统计头部自动机语法）开始，并利用新的判别学习方法来构建可以轻松移植到新数据集的模型。虽然以前的工作通过假设在解析之前有完美的词形消歧来简化问题，但对于大多数语言，准确的词形消歧尚不可用；本项目旨在将形态消歧整合到解析算法中，以提高两项任务的准确性。影响：该项目将通过直接推进超过5亿人使用的语言的核心语言处理技术，以及——由于语言可移植性原则——通过促进对更多语言的未来工作，改善全球信息获取。预期这一项目将提高所考虑的语言的解析精度，所开发的模型和算法将免费提供给研究目的。这些工具有望帮助研究人员从事应用技术，如机器翻译和多语言信息提取。

The field of natural language processing (NLP) has, to date, largely focused its efforts on technology for English, even though it is a typological outlier and the majority of the world's people do not speak it. This project aims to develop statistical natural language analysis tools to disambiguate the morphological and syntactic structure of non-English text. Specifically, the objective of the pilot study is to design, train, implement, and disseminate statistical morpho-syntactic parsing models for Arabic and Hebrew. This project starts with a straightforward formalism (statistical head automaton grammars) and makes use of novel discriminative learning methods to build models that can be easily ported to new datasets. While previous work has simplified the problem by assuming perfect morphological disambiguation prior to parsing, for most languages, accurate morphological disambiguation is not yet available; this project aims to integrate morphological disambiguation into the parsing algorithm for better accuracy on both tasks. Impact: This project will improve global access to information by directly advancing core language processing technology in languages spoken by more than half a billion people and - because of the language-portability principle - by facilitating future work on many more languages. It is expected that this project will improve the state-of-the-art in parsing accuracy for the languages under consideration, and the models and algorithms developed will be made freely available for research purposes. These tools are expected to aid researchers working on applied technologies such as machine translation and multilingual information extraction.