SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics-猫眼课题宝

DOI：

10.18653/v1/2021.findings-acl.10

发表时间：

2021-06

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Hitomi Yanaka;K. Mineshima;Kentaro Inui

最近，深度神经网络 (DNN) 在语义上具有挑战性的 NLP 任务中取得了巨大成功，但仍不清楚 DNN 模型是否可以捕获组合意义，即形式语义学中长期研究的意义方面。为了研究这个问题，我们提出了一个基于自然语言语义（SyGNS）的系统泛化测试平台，其挑战是将自然语言句子映射到多种形式的范围含义表示，旨在解释各种语义现象。使用 SyGNS，我们测试神经网络是否可以系统地解析涉及逻辑表达式（例如量词和否定）的新颖组合的句子。实验表明，Transformer 和 GRU 模型可以推广到看不见的量词、否定和修饰符组合，这些组合在形式上与给定的训练实例相似，但与其他实例不同。我们还发现，当意义表示的形式更简单时，对未见过的组合的泛化性能更好。 SyGNS 的数据和代码可在 https://github.com/verypluming/SyGNS 上公开获取。

Recently, deep neural networks (DNNs) have achieved great success in semantically challenging NLP tasks, yet it remains unclear whether DNN models can capture compositional meanings, those aspects of meaning that have been long studied in formal semantics. To investigate this issue, we propose a Systematic Generalization testbed based on Natural language Semantics (SyGNS), whose challenge is to map natural language sentences to multiple forms of scoped meaning representations, designed to account for various semantic phenomena. Using SyGNS, we test whether neural networks can systematically parse sentences involving novel combinations of logical expressions such as quantifiers and negation. Experiments show that Transformer and GRU models can generalize to unseen combinations of quantifiers, negations, and modifiers that are similar to given training instances in form, but not to the others. We also find that the generalization performance to unseen combinations is better when the form of meaning representations is simpler. The data and code for SyGNS are publicly available at https://github.com/verypluming/SyGNS.

SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics

文献摘要

相似文献