Training Set Debugging Using Trusted Items
Training Set Debugging Using Trusted Items
复制标题
DOI：
10.1609/aaai.v32i1.11610
复制
发表时间：
2018-01
期刊：
影响因子：
--
通讯作者：
Xuezhou Zhang;Xiaojin Zhu;Stephen J. Wright
中科院分区：
文献类型：
--
作者：
Xuezhou Zhang;Xiaojin Zhu;Stephen J. Wright
训练集错误是数据中对机器学习产生不利影响的缺陷。训练集通常太大，无法进行手动检查，但可以有资源来验证一些可信项目。可信项集本身可能不足以进行学习，因此我们提出了一种算法，该算法使用这些项来识别训练集中的错误，从而改善学习。具体来说，我们的方法寻求训练集标签的最小变化集，以便从这个校正的训练集学习的模型正确地预测可信项目的标签。我们将标签被更改的项目标记为潜在的bug，其标签可以由人类专家检查其准确性。在玩具数据和真实的数据上的实验表明，该方法能有效地识别训练集上的错误，并对标签提出适当的修改建议。我们的算法是迈向值得信赖的机器学习的一步。
Training set bugs are flaws in the data that adversely affect machine learning. The training set is usually too large for manual inspection, but one may have the resources to verify a few trusted items. The set of trusted items may not by itself be adequate for learning, so we propose an algorithm that uses these items to identify bugs in the training set and thus improves learning. Specifically, our approach seeks the smallest set of changes to the training set labels such that the model learned from this corrected training set predicts labels of the trusted items correctly. We flag the items whose labels are changed as potential bugs, whose labels can be checked for veracity by human experts. To find the bugs in this way is a challenging combinatorial bilevel optimization problem, but it can be relaxed into a continuous optimization problem.Experiments on toy and real data demonstrate that our approach can identify training set bugs effectively and suggest appropriate changes to the labels. Our algorithm is a step toward trustworthy machine learning.