A Primal Dual Formulation For Deep Learning With Constraints-猫眼课题宝

DOI：

发表时间：

2019

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Yatin Nandwani;A. Pathak;Mausam;Parag Singla

对于几个感兴趣的问题，存在于输出标签空间上的自然约束。例如，对于NER和POS标记的联合任务，这些约束可能指定NER标记“组织”仅与POS标记“名词”和“介词”一致。这些约束可以是将先验知识注入深度学习模型的好方法，从而提高整体性能。在本文中，我们提出了一种约束优化公式，用于训练具有给定输出标签硬约束集的深度网络。我们的新方法首先将标签约束转换为网络输出的概率分布上的软逻辑约束。然后，它将约束优化问题转化为交替的最小-最大优化，为每个约束定义拉格朗日变量。由于约束是独立的目标标签，我们的框架很容易推广到半监督设置。我们在语义角色标注（SRL）、命名实体识别（NER）标注和细粒度实体分类任务上进行了实验，结果表明，我们的约束不仅显著减少了违反约束的次数，而且还可以产生最先进的性能

For several problems of interest, there are natural constraints which exist over the output label space. For example, for the joint task of NER and POS labeling, these constraints might specify that the NER label ‘organization’ is consistent only with the POS labels ‘noun’ and ‘preposition’. These constraints can be a great way of injecting prior knowledge into a deep learning model, thereby improving overall performance. In this paper, we present a constrained optimization formulation for training a deep network with a given set of hard constraints on output labels. Our novel approach first converts the label constraints into soft logic constraints over probability distributions outputted by the network. It then converts the constrained optimization problem into an alternating min-max optimization with Lagrangian variables defined for each constraint. Since the constraints are independent of the target labels, our framework easily generalizes to semi-supervised setting. We experiment on the tasks of Semantic Role Labeling (SRL), Named Entity Recognition (NER) tagging, and fine-grained entity typing and show that our constraints not only significantly reduce the number of constraint violations, but can also result in state-of-the-art performance

A Primal Dual Formulation For Deep Learning With Constraints

文献摘要

被引文献

相似文献