EAGER: Collaborative Research: Toward Informing Users About Algorithmic Fairness

计算机做出关于人的重要决定，包括刑事司法问题，如量刑和保释。如果计算机系统不公平对待人（例如不同种族的人），这些决定有时会被认为是歧视性的。然而，决定一个计算机系统“公平”意味着什么是复杂的：公平有许多可能的数学定义，一个系统不可能同时实现它们。为了让社会制定与这些公平定义相关的政策，非技术人员——从法律和政策专家到普通公众——必须能够理解数学概念之间的微妙区别。本研究将开发和评估向非专家解释这些概念的方法，以便未来的研究可以调查人们对这些概念的看法。拟议的工作将开发和评估文本和图形描述和/或插图，说明不同的非歧视属性及其权衡。具体而言，在这项探索性工作中，该项目将只关注与准确性类似的非歧视性质，仅在刑事司法背景下，例如保释和量刑决定中使用的算法。该项目将采用迭代的、定性的、以人为本的设计，包括与法律和社会科学领域的非计算机科学专家以及外行人进行访谈和共同设计研究，以发展和初步评估这些解释。同时，该项目将对非歧视属性的空间进行系统化。这一努力将为定性设计工作提供信息；与此同时，与法律和伦理专家的访谈也将在一个迭代改进的过程中形成系统化。最终的结果将是描述各种非歧视定义如何沿着实证研究发现最重要的轴不同。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Computers make important decisions about people, including about criminal justice issues such as sentencing and bail. These decisions can sometimes be considered discriminatory if the computer system does not treat people -- for example, people of different races -- fairly. However, deciding what it means for a computer system to be "fair" is complicated: there are many possible mathematical definitions of fairness, and a system cannot achieve them all at the same time. For society to make policy related to these definitions of fairness, non-technical people -- from legal and policy experts to the general public -- must be able to understand subtle distinctions between mathematical concepts. This research will develop and evaluate approaches to explaining these concepts to non-experts, so that future research can investigate people's opinions about them. The proposed work will develop and evaluate text and graphical descriptions and/or vignettes illustrating different nondiscrimination properties and their tradeoffs. For concreteness, in this exploratory work the project will focus only on accuracy-like nondiscrimination properties, only in the context of criminal justice, such as algorithms used in bail and sentencing decisions. The project will use iterative, qualitative, person-centered design, including interviews and co-design studies with both non-computer-science subject-matter experts in law and social science and laypeople to develop and preliminarily evaluate the explanations. In parallel, the project will systematize the space of nondiscrimination properties. This effort will inform qualitative design efforts; concurrently, interviews with legal and ethical experts will also shape the systematization, in a process of iterative refinement. The end product will be a description of how various nondiscrimination definitions differ along the axes empirical studies find most important.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.