Knowledge-Augmented Language Models for Cause-Effect Relation Classification
Knowledge-Augmented Language Models for Cause-Effect Relation Classification
复制标题
DOI：
10.18653/v1/2022.csrr-1.6
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Pedram Hosseini;David A. Broniatowski;Mona T. Diab
中科院分区：
文献类型：
--
作者：
Pedram Hosseini;David A. Broniatowski;Mona T. Diab
以前的研究已经证明了知识增强方法在预训练语言模型中的有效性。但是，这些方法在域和下游任务中的行为不同。在这项工作中，我们研究了在因果关系分类和常识因果推理任务中使用知识图数据增强预训练语言模型。在ATOMIC 2020（一个广泛覆盖的常识推理知识图）中自动描述三元组后，我们继续预训练BERT，并评估因果对分类和回答常识因果推理问题的结果模型。我们的研究结果表明，一个持续预训练的语言模型增强了常识推理知识，在两个常识因果推理基准（COPA和BCOPA-CE）以及时间和因果推理（TCR）数据集上的表现优于我们的基线，而无需对模型架构进行额外改进或使用质量增强的数据进行微调。
Previous studies have shown the efficacy of knowledge augmentation methods in pretrained language models. However, these methods behave differently across domains and downstream tasks. In this work, we investigate the augmentation of pretrained language models with knowledge graph data in the cause-effect relation classification and commonsense causal reasoning tasks. After automatically verbalizing triples in ATOMIC2020, a wide coverage commonsense reasoning knowledge graph, we continually pretrain BERT and evaluate the resulting model on cause-effect pair classification and answering commonsense causal reasoning questions. Our results show that a continually pretrained language model augmented with commonsense reasoning knowledge outperforms our baselines on two commonsense causal reasoning benchmarks, COPA and BCOPA-CE, and a Temporal and Causal Reasoning (TCR) dataset, without additional improvement in model architecture or using quality-enhanced data for fine-tuning.