Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation-猫眼课题宝

DOI：

发表时间：

2021-06

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Cody Blakeney;Nathaniel Huish;Yan Yan-Yan;Ziliang Zong

近年来，人工智能无处不在的部署在算法偏见、歧视和公平性方面引起了极大的关注。与人类造成的传统形式的偏见或歧视相比，人工智能产生的算法偏见更加抽象和不直观，因此更难以解释和减轻。在当前的文献中，关于评估和减轻修剪神经网络中的偏差存在明显的差距。在这项工作中，我们努力解决评估，减轻和解释修剪神经网络中的诱导偏差的挑战性问题。我们的论文有三个贡献。首先，我们提出了两个简单而有效的度量，组合误差方差（CEV）和对称距离误差（EFT），以定量评估修剪模型的诱导偏差预防质量。其次，我们证明了知识蒸馏可以减轻修剪神经网络中的诱导偏差，即使是不平衡的数据集。第三，我们揭示了模型相似性与修剪诱导的偏差有很强的相关性，这为解释为什么修剪神经网络中会出现偏差提供了一个强有力的方法。我们的代码可在https://github.com/codestar12/pruning-distilation-bias上获得

In recent years the ubiquitous deployment of AI has posed great concerns in regards to algorithmic bias, discrimination, and fairness. Compared to traditional forms of bias or discrimination caused by humans, algorithmic bias generated by AI is more abstract and unintuitive therefore more difficult to explain and mitigate. A clear gap exists in the current literature on evaluating and mitigating bias in pruned neural networks. In this work, we strive to tackle the challenging issues of evaluating, mitigating, and explaining induced bias in pruned neural networks. Our paper makes three contributions. First, we propose two simple yet effective metrics, Combined Error Variance (CEV) and Symmetric Distance Error (SDE), to quantitatively evaluate the induced bias prevention quality of pruned models. Second, we demonstrate that knowledge distillation can mitigate induced bias in pruned neural networks, even with unbalanced datasets. Third, we reveal that model similarity has strong correlations with pruning induced bias, which provides a powerful method to explain why bias occurs in pruned neural networks. Our code is available at https://github.com/codestar12/pruning-distilation-bias

Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation

文献摘要

被引文献

相似文献