CAREER: RI: Structural Linguistic Generalization Through Expert-Designed Tasks

作为一种语言的使用者，我们可以理解以前从未读过的复杂句子。我们通过一个叫做结构概括的过程来做到这一点：我们把句子分成更小的部分，解释每个部分的意思，然后把这些意思放在一起来解释整个句子。目前的语言技术，如许多手机上安装的虚拟助手，并不总是能够以这种方式产生意义。这导致他们在被期望理解更复杂的命令时犯错误；这些错误在小社区使用的语言中可能更常见。参与该项目的研究人员将创建多种语言的测试，以衡量计算系统的结构泛化能力，并将使用这些测试来教系统正确泛化。这将导致系统更可靠地工作，并跨越更广泛的语言。除了技术贡献外，该项目还将帮助培养精通人工智能和人类语言结构和多样性的新一代科学家。该项目还将通过向对语言感兴趣的学生介绍人工智能的外展活动吸引新人群进入该领域。从技术角度来看，研究人员将创建专家设计和现实世界的语义解析任务-从自然语言话语映射到捕获其含义的形式语言-这将跨越不同的语言样本。在专家设计的任务中，自然语言输入将由语言学家创建的语法生成，输出将是根据输入计算出的λ演算公式。语言结构的分布在每个任务的训练集和测试集之间会有系统的不同（“结构分裂”）。专家设计任务的结构分裂将用于赋予人工神经网络，如大型语言模型，具有有利于结构泛化的归纳偏差；这样的归纳偏差将导致对现实世界任务的结构泛化。这将使用元学习方法来完成，其目标是从分裂的一部分学习并推广到另一部分。这些行为实验将伴随着对网络内部属性（如初始权重、训练动态和损失情况）如何产生结构泛化的调查。总的来说，通过提供测量和促进跨语言结构泛化的工具，该项目将导致更健壮和样本效率更高的语言理解系统。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As speakers of a language, we can understand complex sentences that we have never read before. We do so through a process called structural generalization: we divide the sentence into smaller parts, interpret the meaning of each part, and then put these meanings together to interpret the full sentence. Current language technologies, such as the virtual assistants installed on many mobile phones, are not always able to generate meaning in this way. This leads them to make mistakes when they are expected to understand more complex commands; these mistakes may be more common in languages that are spoken by smaller communities. The researchers involved in this project will create tests in multiple languages in order to measure a computational system’s structural generalization capabilities, and will use those tests to teach the system to generalize correctly. This will result in systems that work more reliably and across a wider range of languages. Alongside its technological contribution, this project will help train a new generation of scientists that are well-versed in both artificial intelligence and the structure and diversity of human languages. The project will also draw new populations to the field through outreach events that will introduce students interested in language to artificial intelligence.From a technical standpoint, the researchers will create both expert-designed and real-world semantic parsing tasks – mappings from natural language utterances to a formal language that captures their meaning – which will span a diverse sample of languages. In the expert-designed tasks, natural language inputs will be generated from a grammar created by a linguist, and the outputs will be lambda calculus formulas computed from the input. The distribution of linguistic structures will differ systematically between the training and test set of each task (“structural splits”). The structural splits of the expert-designed task will be used to endow artificial neural networks, such as large language models, with an inductive bias that favors structural generalization; such an inductive bias will then lead to structural generalization on real-world tasks. This will be accomplished using a meta-learning approach, whose objective is to learn from one part of the split and generalize to the other. These behavioral experiments will be accompanied by an investigation of how the network’s internal properties, such as initial weights, training dynamics and loss landscape, give rise to structural generalization. Overall, by providing tools for measuring and promoting structural generalization across languages, this project will lead to more robust and sample-efficient language understanding systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.