EAGER: Collaborative Research: Toward Informing Users About Algorithmic Fairness

EAGER：协作研究：向用户通报算法公平性

• 批准号: 1844462
• 负责人: Michelle Mazurek
• 金额: $ 14.73万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1844462&HistoricalAwards=false
• 关键词: EAGER; Collaborative; Research; Toward; Informing

Computers make important decisions about people, including about criminal justice issues such as sentencing and bail. These decisions can sometimes be considered discriminatory if the computer system does not treat people -- for example, people of different races -- fairly. However, deciding what it means for a computer system to be "fair" is complicated: there are many possible mathematical definitions of fairness, and a system cannot achieve them all at the same time. For society to make policy related to these definitions of fairness, non-technical people -- from legal and policy experts to the general public -- must be able to understand subtle distinctions between mathematical concepts. This research will develop and evaluate approaches to explaining these concepts to non-experts, so that future research can investigate people's opinions about them. The proposed work will develop and evaluate text and graphical descriptions and/or vignettes illustrating different nondiscrimination properties and their tradeoffs. For concreteness, in this exploratory work the project will focus only on accuracy-like nondiscrimination properties, only in the context of criminal justice, such as algorithms used in bail and sentencing decisions. The project will use iterative, qualitative, person-centered design, including interviews and co-design studies with both non-computer-science subject-matter experts in law and social science and laypeople to develop and preliminarily evaluate the explanations. In parallel, the project will systematize the space of nondiscrimination properties. This effort will inform qualitative design efforts; concurrently, interviews with legal and ethical experts will also shape the systematization, in a process of iterative refinement. The end product will be a description of how various nondiscrimination definitions differ along the axes empirical studies find most important.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机做出关于人的重要决定，包括关于判刑和保释等刑事司法问题。如果计算机系统不公平地对待人，例如不同种族的人，这些决定有时会被认为是歧视性的。然而，决定计算机系统的“公平”意味着什么是复杂的：公平有许多可能的数学定义，并且系统不能同时实现它们。为了让社会制定与这些公平定义相关的政策，非技术人员--从法律的和政策专家到普通公众--必须能够理解数学概念之间的微妙区别。这项研究将开发和评估向非专家解释这些概念的方法，以便未来的研究可以调查人们对这些概念的看法。拟议的工作将制定和评价文字和图形说明和/或插图，说明不同的不歧视特性及其权衡。具体来说，在这项探索性工作中，该项目将只关注准确性，仅在刑事司法的背景下，如保释和判刑决定中使用的算法。该项目将使用迭代的，定性的，以人为本的设计，包括与法律和社会科学领域的非计算机科学主题专家和外行进行访谈和共同设计研究，以开发和初步评估解释。与此同时，该项目将使不歧视财产的空间系统化。 这项工作将为定性设计工作提供信息;同时，与法律的和伦理专家的访谈也将在迭代改进的过程中形成系统化。 最终产品将是一个描述如何不同的非歧视定义沿着轴实证研究发现最重要的。这个奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的知识价值和更广泛的影响审查标准的支持。

项目成果

Measuring non-expert comprehension of machine learning fairness metrics

衡量非专家对机器学习公平性指标的理解

• 发表时间: 2020
• 期刊: International Conference on Machine Learning (ICML
• 作者: Saha, Debjani;Schumann, Candice;McElfresh, Duncan C.;Dickerson, John P.;Mazurek, Michelle L.;Tschantz, Michael Carl
• 通讯作者: Tschantz, Michael Carl

Human Comprehension of Fairness in Machine Learning

人类对机器学习公平性的理解

• DOI: 10.1145/3375627.3375819
• 发表时间: 2020
• 期刊: and Society
• 作者: Saha, Debjani;Schumann, Candice;McElfresh, Duncan C.;Dickerson, John P.;Mazurek, Michelle L.;Tschantz, Michael Carl
• 通讯作者: Tschantz, Michael Carl