EAGER: Collaborative Research: Toward Informing Users About Algorithmic Fairness

EAGER：协作研究：向用户通报算法公平性

• 批准号: 1844462
• 负责人: Michelle Mazurek
• 金额: $ 14.73万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1844462&HistoricalAwards=false
• 关键词: EAGER; Collaborative; Research; Toward; Informing

Computers make important decisions about people, including about criminal justice issues such as sentencing and bail. These decisions can sometimes be considered discriminatory if the computer system does not treat people -- for example, people of different races -- fairly. However, deciding what it means for a computer system to be "fair" is complicated: there are many possible mathematical definitions of fairness, and a system cannot achieve them all at the same time. For society to make policy related to these definitions of fairness, non-technical people -- from legal and policy experts to the general public -- must be able to understand subtle distinctions between mathematical concepts. This research will develop and evaluate approaches to explaining these concepts to non-experts, so that future research can investigate people's opinions about them. The proposed work will develop and evaluate text and graphical descriptions and/or vignettes illustrating different nondiscrimination properties and their tradeoffs. For concreteness, in this exploratory work the project will focus only on accuracy-like nondiscrimination properties, only in the context of criminal justice, such as algorithms used in bail and sentencing decisions. The project will use iterative, qualitative, person-centered design, including interviews and co-design studies with both non-computer-science subject-matter experts in law and social science and laypeople to develop and preliminarily evaluate the explanations. In parallel, the project will systematize the space of nondiscrimination properties. This effort will inform qualitative design efforts; concurrently, interviews with legal and ethical experts will also shape the systematization, in a process of iterative refinement. The end product will be a description of how various nondiscrimination definitions differ along the axes empirical studies find most important.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机对人做出重要决定，包括量刑和保释等刑事司法问题。如果计算机系统不公平地对待人们（例如不同种族的人），这些决定有时会被视为歧视性的。然而，确定计算机系统的“公平”意味着什么是很复杂的：公平有许多可能的数学定义，而一个系统无法同时实现所有这些定义。为了让社会制定与这些公平定义相关的政策，非技术人员——从法律和政策专家到普通公众——必须能够理解数学概念之间的微妙区别。这项研究将开发和评估向非专家解释这些概念的方法，以便未来的研究可以调查人们对这些概念的看法。拟议的工作将开发和评估文本和图形描述和/或插图，说明不同的非歧视属性及其权衡。具体而言，在这项探索性工作中，该项目将仅关注刑事司法背景下类似准确性的非歧视属性，例如保释和量刑决定中使用的算法。该项目将采用迭代、定性、以人为本的设计，包括与法律和社会科学领域的非计算机科学主题专家以及外行人士进行访谈和共同设计研究，以制定和初步评估解释。与此同时，该项目将使非歧视属性的空间系统化。 这项工作将为定性设计工作提供信息；同时，对法律和道德专家的访谈也将在迭代完善的过程中形成系统化。 最终产品将描述各种非歧视定义在实证研究发现最重要的轴上有何不同。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Measuring non-expert comprehension of machine learning fairness metrics

衡量非专家对机器学习公平性指标的理解

• 发表时间: 2020
• 期刊: International Conference on Machine Learning (ICML
• 作者: Saha, Debjani;Schumann, Candice;McElfresh, Duncan C.;Dickerson, John P.;Mazurek, Michelle L.;Tschantz, Michael Carl
• 通讯作者: Tschantz, Michael Carl

Human Comprehension of Fairness in Machine Learning

人类对机器学习公平性的理解

• DOI: 10.1145/3375627.3375819
• 发表时间: 2020
• 期刊: and Society
• 作者: Saha, Debjani;Schumann, Candice;McElfresh, Duncan C.;Dickerson, John P.;Mazurek, Michelle L.;Tschantz, Michael Carl
• 通讯作者: Tschantz, Michael Carl