When Biased Humans Meet Debiased AI: A Case Study in College Major Recommendation
When Biased Humans Meet Debiased AI: A Case Study in College Major Recommendation
复制标题
DOI：
10.1145/3611313
复制
发表时间：
2023-09-01
影响因子：
3.4
通讯作者：
Pan，Shimei
中科院分区：
文献类型：
--
作者：
Wang，Clarice;Wang，Kathryn;Pan，Shimei
目前，人们对不公平的人工智能(AI)和机器学习(ML)的研究兴趣激增，这些研究旨在减少人工智能算法中的歧视性偏见，例如，性别、年龄和种族。虽然该领域的大多数研究都集中在开发公平的人工智能算法上，但在这项工作中，我们研究了当人类和公平的人工智能交互时出现的挑战。我们的结果表明，由于人类偏好和公平之间的明显冲突，一个公平的人工智能算法本身可能不足以在现实世界中达到预期的结果。以高校专业推荐为例，利用性别去偏向机器学习技术构建了一个公平的人工智能推荐器。我们的线下评估显示，去偏见的推荐者在不牺牲预测准确性的情况下做出了更公平的职业推荐。然而，一项对200多名大学生进行的在线用户研究显示，参与者平均而言更喜欢原始的有偏见的系统，而不是无偏见的系统。具体地说，我们发现，感知到的性别差异是接受一项建议的决定因素。换句话说，如果不解决人类的性别偏见，我们就不能完全解决人工智能建议中的性别偏见问题。我们进行了一项跟踪调查，以获得对各种设计选项的有效性的更多见解，这些选项可以帮助参与者克服自己的偏见。我们的结果表明，让公平的人工智能成为可解释的，对于增加它在现实世界中的采用至关重要。
Currently, there is a surge of interest infair Artificial Intelligence (AI) and Machine Learning (ML)research which aims to mitigate discriminatory bias in AI algorithms, e.g., along lines of gender, age, and race. While most research in this domain focuses on developing fair AI algorithms, in this work, we examine the challenges which arise when humans and fair AI interact. Our results show that due to an apparent conflict between human preferences and fairness, a fair AI algorithm on its own may be insufficient to achieve its intended results in the real world. Using college major recommendation as a case study, we build a fair AI recommender by employing gender debiasing machine learning techniques. Our offline evaluation showed that the debiased recommender makes fairer career recommendations without sacrificing its accuracy in prediction. Nevertheless, an online user study of more than 200 college students revealed that participants on average prefer the original biased system over the debiased system. Specifically, we found that perceived gender disparity is a determining factor for the acceptance of a recommendation. In other words, we cannot fully address the gender bias issue in AI recommendations without addressing the gender bias in humans. We conducted a follow-up survey to gain additional insights into the effectiveness of various design options that can help participants to overcome their own biases. Our results suggest that making fair AI explainable is crucial for increasing its adoption in the real world.