EAGER: PARTIAL: An Exploratory Study on Practical Approaches for Robust NLP Tools with Integrated Annotation Languages

为了开发自然语言处理（NLP）技术，用于更广泛的语言，方言，体裁和风格的文本，这项探索性研究的早期资助研究了一种新的方法。 传统上，语言专家被用来创建黄金标准的语言注释数据集，监督机器学习算法被应用于这些数据集。 这个项目通过将更多的负担转移到学习算法上，将注释者从注释必须完整的要求中解放出来。算法的发展是强大的部分证据，注释器的变化，和噪声由于错误。 因此，任何语言爱好者（不仅仅是受过训练的专家）都可以提供注释，这样就可以用更少的钱为更多语言的更多类型的文本开发NLP。 在这次探索中，重点是依赖分析，这是一个基本的NLP组件，可以预测句子中单词之间的语法关系，并对英语（两种类型），中文和波斯语的数据进行实验。 该方法的正式基础是一个称为图形片段语言（GFL）的框架。 该项目评估了从GFL学习的解析器的质量和注释器的生产力，并赋予了这种新的灵活性。除了对新方法的文档和评估之外，该项目还制作了开源软件工具，用于收集注释数据并使用数据构建NLP工具。 它强调了这些工具在课堂上的可用性，提供了可以在NLP和语言学课程中使用的练习，让学生直接参与数据，使用数据的模型，以及数据注释支持的技术目标。

In order to develop natural language processing (NLP) technologies for text in a wider range of languages, dialects, genres, and styles, this Early Grant for Exploratory Research investigates a novel methodological approach. Conventionally, linguistic experts are employed to create gold-standard linguistically annotated datasets to which supervised machine learning algorithms are applied. This project frees annotators from the requirement that annotations be complete by moving more of the burden to learning algorithms. Algorithms are developed that are robust to partial evidence, annotator variation, and noise due to errors. As a result, any language enthusiast (not just trained experts) can provide annotations so that NLP can be developed for more kinds of text in more languages for less money. In this exploration, the focus is on dependency parsing, a fundamental NLP component that predicts the grammatical relationships between words in sentences, with experimentation on data in English (two genres), Chinese, and Farsi. The formal basis for the approach is a framework called Graph Fragment Language (GFL). The project assesses the quality of parsers learned from GFL and the productivity of annotators accorded this new flexibility.Beyond documentation and assessment of the new methodology, this project produces open-source software tools for gathering annotated data and constructing NLP tools using the data. It emphasizes the usability of these tools in classrooms, contributing exercises that can be used in NLP and linguistics courses to allow students to engage directly with data, with the models that make use of the data, and with the technological goals that data annotation supports.