Fairness without Demographics through Knowledge Distillation
Fairness without Demographics through Knowledge Distillation
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Junyi Chai;T. Jang;Xiaoqian Wang
中科院分区：
文献类型：
--
作者：
Junyi Chai;T. Jang;Xiaoqian Wang
大多数关于公平性的现有工作都假设训练集中有可用的人口统计信息。在实践中，由于法律的或隐私问题，当人口统计信息在训练集中不可用时，找到替代目标以确保公平性至关重要。现有的工作公平没有人口统计遵循罗尔斯最大最小公平目标。然而，这样的约束可能过于严格，以提高群体公平性，并可能导致准确性大大降低。针对这些局限性，本文提出从一个新的角度来解决这一问题，即，通过知识的升华。我们的方法使用来自过拟合教师模型的软标签作为替代方案，并且我们从初步实验中表明，软标签有利于提高公平性。我们从理论上分析了我们的方法的公平性，我们表明，我们的方法可以被视为一个基于错误的重新加权。三个数据集上的实验结果表明，我们的方法优于国家的最先进的替代品，具有显着的改善组公平性和相对较小的准确性下降。
Most of existing work on fairness assumes available demographic information in the training set. In practice, due to legal or privacy concerns, when demographic information is not available in the training set, it is crucial to find alternative objectives to ensure fairness. Existing work on fairness without demographics follows Rawlsian Max-Min fairness objectives. However, such constraints could be too strict to improve group fairness, and could lead to a great decrease in accuracy. In light of these limitations, in this paper, we propose to solve the problem from a new perspective, i.e., through knowledge distillation. Our method uses soft label from an overfitted teacher model as an alternative, and we show from preliminary experiments that soft labelling is beneficial for improving fairness. We analyze theoretically the fairness of our method, and we show that our method can be treated as an error-based reweighing. Experimental results on three datasets show that our method outperforms state-of-the-art alternatives, with notable improvements in group fairness and with relatively small decrease in accuracy.