CRII: RI: Can Low-Bias Machine Learners Acquire English Grammar? Deep Learning and Linguistic Acceptability

广泛部署的语言技术应用，如翻译系统和智能助手，在很大程度上依赖于机器学习模型来理解句子。这些模型从数据中学习理解语言，这些数据通常像出版的书籍集合或维基百科的下载一样简单，而不是通过任何形式的人工工程或语言专家的实际指导。虽然现代机器学习方法非常有效，但它们并不完美。当他们无法理解某些文本时，可能很难发现原因，甚至更难制定干预措施来解决这些失败。这个CISE研究启动计划（CRII）项目开发工具来帮助使用语言科学研究的方法和见解来分析和改进句子理解的机器学习系统。该项目应该在使开发有效的语言技术变得更容易方面产生实际影响，在帮助语言学家使用机器学习作为研究人类语言学习的代理方面产生科学影响，并在通过研究研讨会和直接研究合作支持几名博士生方面产生培训影响，因为他们将发展成为语言科学和语言技术之间相互作用的专家。该项目中使用的方法依赖于人类判断句子语法可接受性的能力；也就是说，决定某人是否可以使用给定的单词序列来表达某事。该项目有三个部分：(1)建立一个基于可接受性的大型英语数据集，评估机器学习系统的语言知识；(2)使用这些数据来评估广泛使用的语言机器学习标准方法，重点关注最近使用人工神经网络从纯文本中学习的有前途的方法；(3)开发使用小型自定义数据集的方法，直接修复这些机器学习模型获得的知识中的任何空白。分析和改进人工神经网络是困难的，因为它们对语言的内部表示是连续的，至少从表面上看，它们对语言的内部表示与语言学家用来分析语言的那种表示根本不相似。研究人员的方法旨在最大限度地减少这种困难，这种方法依赖于在实验中使用相同数据的多种方法收集证据。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Widely-deployed applications of language technology such as translation systems and smart assistants rely heavily on machine learning models for sentence understanding. These models learn to understand language from data, which can often be as simple as a collection of published books or a download of Wikipedia, rather than through any kind of manual engineering or hands-on guidance by linguistic expert. While modern machine learning methods are quite effective, they are not perfect. When they fail understand some text, it can be difficult to discover why, and even more difficult to craft interventions to address those failures. This CISE Research Initiation Initiative (CRII) project develops tools to help use methods and insights from research in linguistic science to analyze and refine machine learning systems for sentence understanding. The project should have a practical impact in making it easier to develop effective language technologies, a scientific impact in helping linguists use machine learning as a proxy to study human language learning, and a training impact in supporting several PhD students---through both research seminars and direct research collaborations---as they develop into experts in the interaction between linguistic science and language technology.The methods used in this project relies on the human ability to judge the grammatical acceptability of a sentence; i.e., to decide whether someone could ever use a given sequence of words to say something. The project has three parts: (1) to build a large acceptability-based dataset for English which evaluates machine learning systems on their linguistic knowledge; (2) to use this data to evaluate widely-used standard approaches to machine learning for language, with a focus on promising recent approaches that use artificial neural networks learn from plain text; and (3) to develop methods for using small custom datasets to directly repair any gaps in the knowledge that these machine learning models acquire. Analyzing and improving artificial neural networks is difficult, since their internal representations of language are continuous and at least superficially, their internal representations of language do not at all resemble the kinds of representations that linguists use to analyze language. The investigators' methods are designed to minimize this difficulty, which rely on converging evidence from multiple ways of using the same data in its experiments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.