FAI: End-To-End Fairness for Algorithm-in-the-Loop Decision Making in the Public Sector

FAI：公共部门算法在环决策的端到端公平性

• 批准号: 2040898
• 负责人: Daniel Neill
• 金额: $ 62.5万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2040898&HistoricalAwards=false
• 关键词: FAI; End; Fairness; Algorithm; Loop

The goal of this project is to develop methods and tools that assist public sector organizations with fair and equitable policy interventions. In areas such as housing and criminal justice, critical decisions that impact lives, families, and communities are made by a variety of actors, including city officials, police, and court judges. In these high-stakes contexts, human decision makers’ implicit biases can lead to disparities in outcomes across racial, gender, and socioeconomic lines. While artificial intelligence (AI) offers great promise for identifying and potentially correcting these sorts of biases, a rapidly growing literature has shown that automated decision tools can also worsen existing disparities or create new biases. To help bridge this gap between the promise and practice of AI, the interdisciplinary team of investigators will develop an integrated framework and new methodological approaches to support fair and equitable decision-making. This framework is motivated by three main ideas: (1) identifying and mitigating the impacts of biases on downstream decisions and their impacts, instead of simply measuring biases in data and in predictive models; (2) enabling the combination of an algorithmic decision support tool and a human decision-maker to make fairer and more equitable decisions than either human or algorithm alone; and (3) developing operational definitions of fairness and quantitative assessments of bias, guided by stakeholder discussions, that are directly relevant and applicable to the housing and criminal justice domains. The ultimate impact of this work is to advance social justice for those who live in cities, and who rely on city services or are involved with the justice system, by assessing and mitigating biases in decision-making processes and reducing disparities.The project team will address both the risks and the benefits of algorithmic decision-making through transformative technical contributions. First, they will develop a new, pipelined conceptualization of fairness consisting of seven distinct stages: data, models, predictions, recommendations, decisions, impacts, and outcomes. This end-to-end fairness pipeline will account for multiple sources of bias, model how biases propagate through the pipeline to result in inequitable outcomes and assess sensitivity to unmeasured biases. Second, they will build a general methodological framework for identifying and correcting biases at each stage of this pipeline, assessing intersectional and contextual biases across multiple data dimensions, and incorporating new ideas for model assessment and analysis of heterogeneous treatment effects. This generalized bias scan will provide essential information throughout the end-to-end fairness pipeline, informing not only what human and algorithmic biases exist, but what interventions are likely to mitigate these biases. Third, the project addresses algorithm-in-the-loop decision processes, in which an algorithmic decision support tool provides recommendations to a human decision-maker. The investigators will develop approaches for modeling systematic biases in human decisions, identifying possible explanatory factors for those biases, and optimizing individualized algorithmic "nudges" to guide human decisions toward fairness. Finally, the project team will create new metrics for measuring the presence and extent of bias. The outputs of the project will be designed for integration into the operational decision-making of city agencies responsible for making fair and equitable decisions in the criminal justice and housing domains. The investigators will assess the fairness of existing practices and create open-source tools for assessing and correcting biases, for users in each domain. They will develop tools which can be used to (a) reduce incarceration by equitably providing supportive interventions to justice-involved populations; (b) prioritize housing inspections and repairs; (c) assess and improve the fairness of civil and criminal court proceedings; and (d) analyze the disparate health impacts of adverse environmental exposures, including poor-quality housing and aggressive, unfair policing practices. Operational deployments of the developed tools will be regularly and comprehensively evaluated to assess impacts and to avoid unintended consequences, both maximizing the benefits and minimizing potential harms from both algorithmic and human decisions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目标是开发方法和工具，帮助公共部门组织进行公平和公正的政策干预。在住房和刑事司法等领域，影响生活、家庭和社区的关键决策是由各种行动者做出的，包括城市官员、警察和法院法官。在这些高风险的环境中，人类决策者的隐性偏见可能导致不同种族、性别和社会经济阶层的结果差异。虽然人工智能（AI）为识别和纠正这类偏见提供了巨大的希望，但越来越多的文献表明，自动化决策工具也可能加剧现有的差距或产生新的偏见。为了帮助弥合人工智能的前景与实践之间的差距，跨学科研究团队将开发一个综合框架和新的方法方法，以支持公平公正的决策。该框架的动机主要有三个方面：(1)识别和减轻偏差对下游决策及其影响的影响，而不是简单地测量数据和预测模型中的偏差；(2)使算法决策支持工具和人类决策者的结合能够比单独使用人工或算法做出更公平和更公平的决策；(3)在利益相关者讨论的指导下，制定与住房和刑事司法领域直接相关并适用的公平的操作定义和偏见的定量评估。这项工作的最终影响是，通过评估和减轻决策过程中的偏见和缩小差距，为生活在城市、依赖城市服务或参与司法系统的人促进社会正义。项目团队将通过变革性的技术贡献来处理算法决策的风险和收益。首先，他们将开发一种新的、流水线式的公平概念，包括七个不同的阶段：数据、模型、预测、建议、决策、影响和结果。这种端到端公平管道将解释多种偏差来源，模拟偏差如何通过管道传播从而导致不公平的结果，并评估对未测量偏差的敏感性。其次，他们将建立一个通用的方法框架，用于识别和纠正该管道的每个阶段的偏差，评估跨多个数据维度的交叉和上下文偏差，并纳入模型评估和分析异质治疗效果的新思路。这种广义偏见扫描将在整个端到端公平管道中提供基本信息，不仅告知存在哪些人为和算法偏见，还告知哪些干预措施可能减轻这些偏见。第三，该项目解决了循环算法决策过程，其中算法决策支持工具向人类决策者提供建议。研究人员将开发方法来模拟人类决策中的系统性偏见，确定这些偏见的可能解释因素，并优化个性化的算法“推动”，以指导人类决策走向公平。最后，项目团队将创建新的度量标准来度量偏差的存在和程度。该项目的产出将纳入负责在刑事司法和住房领域作出公正和公平决定的城市机构的业务决策。调查人员将评估现有做法的公平性，并为每个领域的用户创建评估和纠正偏见的开源工具。他们将开发可用于(a)通过公平地向参与司法的人群提供支持性干预措施来减少监禁的工具；(b)优先检查和修理房屋；(c)评估和改进民事和刑事法庭诉讼程序的公正性；(d)分析不良环境暴露对健康的不同影响，包括低质量的住房和激进的、不公平的警务做法。开发工具的操作部署将定期进行全面评估，以评估影响并避免意外后果，从而最大限度地提高算法和人为决策的收益并最大限度地减少潜在危害。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Pretrial release judgments and decision fatigue

审前释放判决和决策疲劳

• 发表时间: 2022
• 期刊: Judgment and decision making
• 影响因子: 2.5
• 作者: Shroff, Ravi;Vamvourellis, Konstantinos
• 通讯作者: Vamvourellis, Konstantinos

Provable detection of propagating sampling bias in prediction models

预测模型中传播采样偏差的可证明检测

• 发表时间: 2023
• 期刊: Proceedings of the 37th AAAI Conference on Artificial Intelligence
• 作者: Ravishankar, Pavan;Mo, Qingyu;McFowland III, Edward;Neill, Daniel B.
• 通讯作者: Neill, Daniel B.

Efficient Optimization of Partition Scan Statistics via the Consecutive Partitions Property

通过连续分区属性有效优化分区扫描统计

• DOI: 10.1080/10618600.2022.2077351
• 发表时间: 2022
• 期刊: Journal of Computational and Graphical Statistics
• 影响因子: 2.4
• 作者: Pehlivanian, Charles A.;Neill, Daniel B.
• 通讯作者: Neill, Daniel B.

Commentary on “Causal Decision Making and Causal Effect Estimation Are Not the Same… and Why It Matters”

对“因果决策和因果效应估计不同”及其重要性的评论

• DOI: 10.1287/ijds.2021.0010
• 发表时间: 2022
• 期刊: INFORMS Journal on Data Science
• 作者: McFowland, Edward

The measure and mismeasure of fairness

公平的衡量与误衡量

• 发表时间: 2023
• 期刊: Journal of machine learning research
• 影响因子: 6
• 作者: Corbett-Davies, Sam;Gaebler, Johann D.;Nilforoshan, Hamed;Shroff, Ravi;Goel, Sharad
• 通讯作者: Goel, Sharad