Learning Constraints for Structured Prediction Using Rectifier Networks
Learning Constraints for Structured Prediction Using Rectifier Networks
复制标题
DOI：
10.18653/v1/2020.acl-main.438
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Xingyuan Pan;Maitrey Mehta;Vivek Srikumar
中科院分区：
文献类型：
--
作者：
Xingyuan Pan;Maitrey Mehta;Vivek Srikumar
各种自然语言处理任务都是结构化预测问题，其中输出是由多个相互依赖的决策构建的。过去的工作表明，作为输出空间约束的领域知识可以帮助提高预测准确性。然而，设计良好的约束通常依赖于领域专业知识。在本文中，我们研究学习此类约束的问题。我们将问题描述为训练两层整流器网络以识别有效结构或子结构的问题，并展示了将经过训练的网络转换为推理变量线性约束系统的结构。我们在几个 NLP 任务上的实验表明，学习到的约束可以提高预测精度，特别是当训练样本数量较少时。
Various natural language processing tasks are structured prediction problems where outputs are constructed with multiple interdependent decisions. Past work has shown that domain knowledge, framed as constraints over the output space, can help improve predictive accuracy. However, designing good constraints often relies on domain expertise. In this paper, we study the problem of learning such constraints. We frame the problem as that of training a two-layer rectifier network to identify valid structures or substructures, and show a construction for converting a trained network into a system of linear constraints over the inference variables. Our experiments on several NLP tasks show that the learned constraints can improve the prediction accuracy, especially when the number of training examples is small.