FAI: Organizing Crowd Audits to Detect Bias in Machine Learning

FAI：组织群体审计以检测机器学习中的偏差

• 批准号: 2040942
• 负责人: Jason Hong
• 金额: $ 62.5万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2040942&HistoricalAwards=false
• 关键词: FAI; Organizing; Crowd; Audits; Detect

Machine learning development teams often struggle to detect and mitigate harmful stereotypes due to their own blind spots, particularly when ML systems are deployed globally. These kinds of representation harms cannot be easily quantified using today’s automated techniques or fairness metrics, and require knowledge of specific social, cultural, and historical contexts. The researchers team will develop a crowd audit service that harnesses the power of volunteers and crowd workers to identify specific cases of bias and unfairness in machine learning systems, generalize those to systematic failures, and synthesize and prioritize these findings in a form that is readily actionable by development teams. Success in the research team’s work will lead to new ways to identify bias and unfairness in machine learning systems, thus improving trust and reliability in these systems. The research team’s work will be shared through a public web site that will make it easy for journalists, policy makers, researchers, and the public at large to engage in understanding algorithmic bias as well as participating in finding unfair behaviors in machine learning systems. This project will explore three major research questions. The first is investigating new techniques for recruiting and incentivizing participation from a diverse crowd. The second is developing new and effective forms of guidance for crowd workers for finding instances and generalizing instances of bias. The third is designing new ways of synthesizing findings from the crowd so that development teams can understand and productively act on. The outputs of this research will include developing a taxonomy of harms; designing and evaluating new kinds of tools to help the crowd tag, discuss, and generalize representation harms; synthesizing new design practices in algorithmic socio-technical platforms in which these platforms can provide users with the opportunity to identify and report observed unfair system behaviors via the platform itself; and gathering new data sets consisting of unfair ML system behaviors identified by the crowd. These datasets will support future research into the design of crowd auditing systems, the nature of representation harms in ML systems, and for future ML teams working on similar kinds of systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习开发团队经常因为自己的盲点而难以检测和减轻有害的刻板印象，特别是当机器学习系统在全球部署时。使用当今的自动化技术或公平指标，这些类型的代表性伤害无法轻易量化，并且需要特定的社会、文化和历史背景知识。研究团队将开发一种群体审计服务，利用志愿者和群体工作人员的力量来识别机器学习系统中的偏见和不公平的具体案例，将其归纳为系统故障，并以开发团队易于操作的形式综合和优先考虑这些发现。研究小组工作的成功将导致识别机器学习系统中的偏见和不公平的新方法，从而提高这些系统的信任和可靠性。研究小组的工作将通过一个公共网站分享，这将使记者、政策制定者、研究人员和广大公众更容易理解算法偏见，并参与发现机器学习系统中的不公平行为。这个项目将探讨三个主要的研究问题。首先是研究招募和激励不同人群参与的新技术。第二是为群体工作者提供新的有效指导形式，以发现偏见实例并归纳偏见实例。第三是设计新的方法来综合来自人群的发现，以便开发团队能够理解并有效地采取行动。这项研究的产出将包括制定危害分类；设计和评估新类型的工具，以帮助人群标记、讨论和概括表征危害；在算法社会技术平台中综合新的设计实践，这些平台可以为用户提供通过平台本身识别和报告观察到的不公平系统行为的机会；并收集由人群识别的不公平ML系统行为组成的新数据集。这些数据集将支持未来对人群审计系统设计的研究，ML系统中表示危害的本质，以及未来在类似系统上工作的ML团队。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Understanding Frontline Workers’ and Unhoused Individuals’ Perspectives on AI Used in Homeless Services

• DOI: 10.1145/3544548.3580882
• 发表时间: 2023-03
• 期刊: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
• 作者: Tzu-Sheng Kuo;Hong Shen;Jisoo Geum;N. Jones;Jason I. Hong;Haiyi Zhu;Kenneth Holstein

"Give Everybody [..] a Little Bit More Equity": Content Creator Perspectives and Responses to the Algorithmic Demonetization of Content Associated with Disadvantaged Groups

“给每个人[..]多一点公平”：内容创作者对弱势群体相关内容的算法非货币化的看法和回应

• DOI: 10.1145/3555149
• 发表时间: 2022
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: Kingsley, Sara;Sinha, Proteeti;Wang, Clara;Eslami, Motahhare;Hong, Jason I.
• 通讯作者: Hong, Jason I.

Everyday Algorithm Auditing: Understanding the Power of Everyday Users in Surfacing Harmful Algorithmic Behaviors

日常算法审计：了解日常用户在发现有害算法行为方面的力量

• DOI: 10.1145/3479577
• 发表时间: 2021
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: Shen, Hong;DeVos, Alicia;Eslami, Motahhare;Holstein, Kenneth
• 通讯作者: Holstein, Kenneth

Toward User-Driven Algorithm Auditing: Investigating users’ strategies for uncovering harmful algorithmic behavior

迈向用户驱动的算法审计：调查用户发现有害算法行为的策略

• DOI: 10.1145/3491102.3517441
• 发表时间: 2022
• 期刊: CHI '22: CHI Conference on Human Factors in Computing Systems
• 作者: DeVos, Alicia;Dhabalia, Aditi;Shen, Hong;Holstein, Kenneth;Eslami, Motahhare
• 通讯作者: Eslami, Motahhare

Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work together to Surface Algorithmic Harms?

用户驱动的算法审计的参与和分工：日常用户如何共同揭露算法危害？

• DOI: 10.1145/3544548.3582074
• 发表时间: 2023
• 期刊: ACM
• 作者: Li, Rena;Kingsley, Sara;Fan, Chelsea;Sinha, Proteeti;Wai, Nora;Lee, Jaimie;Shen, Hong;Eslami, Motahhare;Hong, Jason
• 通讯作者: Hong, Jason