EAGER: Collaborative Research: Toward Informing Users About Algorithmic Fairness

计算机做出关于人的重要决定，包括关于刑事司法问题的决定，如量刑和保释。如果计算机系统不公平地对待人们--例如，不同种族的人--这些决定有时可以被认为是歧视性的。然而，决定计算机系统“公平”的含义是复杂的：公平有许多可能的数学定义，一个系统不可能同时实现所有这些定义。为了让社会制定与这些公平定义相关的政策，非技术人员--从法律和政策专家到普通公众--必须能够理解数学概念之间的微妙区别。这项研究将开发和评估向非专家解释这些概念的方法，以便未来的研究可以调查人们对这些概念的看法。拟议的工作将开发和评估文本和图形描述和/或插图，说明不同的非歧视特性及其权衡。具体地说，在这项探索性工作中，该项目将只关注类似准确性的非歧视性质，仅在刑事司法的背景下，例如在保释和量刑决定中使用的算法。该项目将使用迭代的、定性的、以人为中心的设计，包括与法律和社会科学领域的非计算机科学主题专家以及非计算机科学领域的专家进行访谈和共同设计研究，以制定和初步评估解释。同时，该项目将使非歧视性质的空间系统化。这一努力将为质量设计工作提供信息；同时，与法律和伦理专家的访谈也将在反复完善的过程中形成系统化。最终成果将是对各种非歧视定义在实证研究认为最重要的轴线上的不同之处的描述。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computers make important decisions about people, including about criminal justice issues such as sentencing and bail. These decisions can sometimes be considered discriminatory if the computer system does not treat people -- for example, people of different races -- fairly. However, deciding what it means for a computer system to be "fair" is complicated: there are many possible mathematical definitions of fairness, and a system cannot achieve them all at the same time. For society to make policy related to these definitions of fairness, non-technical people -- from legal and policy experts to the general public -- must be able to understand subtle distinctions between mathematical concepts. This research will develop and evaluate approaches to explaining these concepts to non-experts, so that future research can investigate people's opinions about them. The proposed work will develop and evaluate text and graphical descriptions and/or vignettes illustrating different nondiscrimination properties and their tradeoffs. For concreteness, in this exploratory work the project will focus only on accuracy-like nondiscrimination properties, only in the context of criminal justice, such as algorithms used in bail and sentencing decisions. The project will use iterative, qualitative, person-centered design, including interviews and co-design studies with both non-computer-science subject-matter experts in law and social science and laypeople to develop and preliminarily evaluate the explanations. In parallel, the project will systematize the space of nondiscrimination properties. This effort will inform qualitative design efforts; concurrently, interviews with legal and ethical experts will also shape the systematization, in a process of iterative refinement. The end product will be a description of how various nondiscrimination definitions differ along the axes empirical studies find most important.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.