How do the existing fairness metrics and unfairness mitigation algorithms contribute to ethical learning analytics?
How do the existing fairness metrics and unfairness mitigation algorithms contribute to ethical learning analytics?
复制标题
DOI：
10.1111/bjet.13217
复制
发表时间：
2022-04-12
影响因子：
6.6
通讯作者：
Thuc Duy Le
中科院分区：
文献类型：
--
作者：
Deho, Oscar Blessed;Zhan, Chen;Thuc Duy Le
随着学习分析（LA）的广泛使用，关于公平性的伦理问题已经提出。研究表明，LA模型可能对某些人口统计学亚组的学生有偏见。尽管在过去的十年中，公平性在更广泛的机器学习（ML）社区中得到了极大的关注，但直到最近，洛杉矶才开始关注公平性。此外，关于在特定上下文中使用哪种不公平缓解算法或度量的决定在很大程度上仍然是未知的。在此前提下，我们对一些在公平ML社区中被认为是有希望的结果的选择的不公平缓解算法进行了比较评估。使用3年的程序辍学数据从澳大利亚的大学，我们比较评估如何不公平缓解算法有助于道德LA通过测试的一些假设在公平性和性能指标。有趣的是，我们的研究结果表明，数据偏差并不总是导致预测偏差。也许并不奇怪，我们对公平-效用权衡的测试表明，确保公平并不总是导致效用下降。事实上，我们的研究结果表明，在特定情况下，确保公平可能会提高效用。我们的研究结果可能在一定程度上，指导公平算法和度量选择给定的上下文。
With the widespread use of learning analytics (LA), ethical concerns about fairness have been raised. Research shows that LA models may be biased against students of certain demographic subgroups. Although fairness has gained significant attention in the broader machine learning (ML) community in the last decade, it is only recently that attention has been paid to fairness in LA. Furthermore, the decision on which unfairness mitigation algorithm or metric to use in a particular context remains largely unknown. On this premise, we performed a comparative evaluation of some selected unfairness mitigation algorithms regarded in the fair ML community to have shown promising results. Using a 3-year program dropout data from an Australian university, we comparatively evaluated how the unfairness mitigation algorithms contribute to ethical LA by testing for some hypotheses across fairness and performance metrics. Interestingly, our results show how data bias does not always necessarily result in predictive bias. Perhaps not surprisingly, our test for fairness-utility tradeoff shows how ensuring fairness does not always lead to drop in utility. Indeed, our results show that ensuring fairness might lead to enhanced utility under specific circumstances. Our findings may to some extent, guide fairness algorithm and metric selection for a given context.