EAGER: PARTIAL: An Exploratory Study on Practical Approaches for Robust NLP Tools with Integrated Annotation Languages

为了开发更广泛的语言、方言、体裁和风格文本的自然语言处理（NLP）技术，本探索性研究早期拨款研究了一种新的方法方法。传统上，语言专家被雇用来创建黄金标准的语言注释数据集，并应用监督机器学习算法。该项目将更多的负担转移到学习算法上，从而使注释者不必完成注释。我们开发的算法对部分证据、注释者变化和错误引起的噪声具有鲁棒性。因此，任何语言爱好者（不仅仅是训练有素的专家）都可以提供注释，这样就可以用更少的钱为更多语言的更多类型的文本开发NLP。在这个探索中，重点是依赖解析，这是一个基本的NLP组件，预测句子中单词之间的语法关系，并对英语（两种体裁）、汉语和波斯语的数据进行实验。该方法的形式化基础是一个称为图形片段语言（GFL）的框架。该项目评估了从GFL学习的解析器的质量，以及赋予这种新的灵活性的注释器的生产力。除了新方法的文档和评估之外，该项目还生产用于收集注释数据和使用数据构建NLP工具的开源软件工具。它强调了这些工具在课堂上的可用性，提供了可以在NLP和语言学课程中使用的练习，让学生直接接触数据，使用数据的模型，以及数据注释支持的技术目标。

In order to develop natural language processing (NLP) technologies for text in a wider range of languages, dialects, genres, and styles, this Early Grant for Exploratory Research investigates a novel methodological approach. Conventionally, linguistic experts are employed to create gold-standard linguistically annotated datasets to which supervised machine learning algorithms are applied. This project frees annotators from the requirement that annotations be complete by moving more of the burden to learning algorithms. Algorithms are developed that are robust to partial evidence, annotator variation, and noise due to errors. As a result, any language enthusiast (not just trained experts) can provide annotations so that NLP can be developed for more kinds of text in more languages for less money. In this exploration, the focus is on dependency parsing, a fundamental NLP component that predicts the grammatical relationships between words in sentences, with experimentation on data in English (two genres), Chinese, and Farsi. The formal basis for the approach is a framework called Graph Fragment Language (GFL). The project assesses the quality of parsers learned from GFL and the productivity of annotators accorded this new flexibility.Beyond documentation and assessment of the new methodology, this project produces open-source software tools for gathering annotated data and constructing NLP tools using the data. It emphasizes the usability of these tools in classrooms, contributing exercises that can be used in NLP and linguistics courses to allow students to engage directly with data, with the models that make use of the data, and with the technological goals that data annotation supports.