Generating Labeled Data for Relation Extraction: A Meta Learning Approach with Joint GPT-2 Training
Generating Labeled Data for Relation Extraction: A Meta Learning Approach with Joint GPT-2 Training
复制标题
DOI：
10.18653/v1/2023.findings-acl.727
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Amir Pouran Ben Veyseh;Franck Dernoncourt;Bonan Min;Thien Huu Nguyen
中科院分区：
文献类型：
--
作者：
Amir Pouran Ben Veyseh;Franck Dernoncourt;Bonan Min;Thien Huu Nguyen
关系抽取（RelationExtraction，RE）是识别文本中实体之间的语义关系的任务。尽管RE研究取得了重大进展，但RE仍然面临的一个挑战是缺乏用于数据饥渴的深度学习模型的训练数据。标注的成本和任务的难度是在不同领域收集大规模RE数据集的障碍之一。为了解决这个问题，我们提出了一个新的框架来自动生成标记的数据RE。我们的框架提供了用于数据生成的预训练语言模型GPT-2。此外，为了优化RE模型生成的样本，我们引入了一种Meta学习方法，允许GPT-2模型在RE的训练过程中更新。特别是，为了利用来自RE模型的反馈来改进GPT-2的数据生成，我们提出了一种新的奖励函数来用REINFORCE更新GPT-2模型，试图促进为生成的数据和Meta开发集计算的RE损失函数梯度的相似性。我们对两个基准数据集进行了广泛的实验，以产生最先进的RE性能。
Relation Extraction (RE) is the task of identifying semantic relation between real-world entities mentioned in text. Despite significant progress in RE research, a remaining challenge for RE concerns the lack of training data for data-hungry deep learning models. Cost of annotation and difficulty of the task are among hindrance to collect a large-scale RE dataset in different domains. To address this limitation, we propose a novel framework to automatically generate labeled data for RE. Our framework presents the pre-trained language model GPT-2 for data generation. In addition, to optimize the generated samples for an RE model, we introduce a meta learning approach to allow the GPT-2 model to be updated during the training process for RE. In particular, to leverage the feedback from the RE model to improve the data generation from GPT-2, we propose a novel reward function to update the GPT-2 model with REINFORCE, seeking to promote the similarity of the RE loss function’s gradients computed for generated data and a meta development set. We conduct extensive experiments on two benchmark datasets to produce state-of-the-art performance for RE.