Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment
Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment
复制标题
DOI：
--
复制
发表时间：
2018-04
期刊：
影响因子：
--
通讯作者：
Masatoshi Tsuchiya
中科院分区：
文献类型：
--
作者：
Masatoshi Tsuchiya
训练数据的质量是采用以学习为中心的方法时的关键问题之一。本文提出了一种新的方法来调查一个大型语料库的质量设计识别文本蕴涵（RTE）的任务。所提出的方法受到统计假设检验的启发，由两个阶段组成：第一阶段是引入文本蕴涵标签的可预测性作为零假设，如果目标语料库没有隐藏的偏见，则这是极其不可接受的，第二阶段是使用朴素贝叶斯模型来测试零假设。在斯坦福大学自然语言推理（SNLI）语料库上的实验结果并不拒绝零假设。因此，它表明，SNLI语料库有一个隐藏的偏见，允许预测的文本蕴涵标签的假设句，即使没有上下文信息是由前提句。本文还介绍了这种隐藏的偏见造成的RTE的NN模型的性能影响。
The quality of training data is one of the crucial problems when a learning-centered approach is employed. This paper proposes a new method to investigate the quality of a large corpus designed for the recognizing textual entailment (RTE) task. The proposed method, which is inspired by a statistical hypothesis test, consists of two phases: the first phase is to introduce the predictability of textual entailment labels as a null hypothesis which is extremely unacceptable if a target corpus has no hidden bias, and the second phase is to test the null hypothesis using a Naive Bayes model. The experimental result of the Stanford Natural Language Inference (SNLI) corpus does not reject the null hypothesis. Therefore, it indicates that the SNLI corpus has a hidden bias which allows prediction of textual entailment labels from hypothesis sentences even if no context information is given by a premise sentence. This paper also presents the performance impact of NN models for RTE caused by this hidden bias.