BIGDATA: Collaborative Research: F: Algorithmic Fairness: A Systemic and Foundational Treatment of Nondiscriminatory Data Mining

BIGDATA：协作研究：F：算法公平性：非歧视性数据挖掘的系统性和基础性处理

• 批准号: 1633724
• 负责人: Suresh Venkatasubramanian
• 金额: $ 48.41万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2020-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1633724&HistoricalAwards=false
• 关键词: BIGDATA; Collaborative; Research; Algorithmic; Fairness

Data-driven modeling has moved beyond the realm of consumer predictions and recommendations into areas of policy and planning that have a profound impact on our daily lives. The tools of data analysis are being harnessed to predict crime, select candidates for jobs, identify security threats, determine credit risk, and even decide treatment plans and interventions for patients. Automated learning and mining tools can crunch incredible amounts and variety of data in order to detect patterns and make predictions. As is rapidly becoming clear, these tools can also introduce discriminatory behavior and amplify biases in the systems they are trained on. In this project, the PIs will study the problems of discrimination and bias in algorithmic decision-making. By studying all aspects of the data pipeline (from data preparation to learning, evaluation, and feedback), they will develop tools for analyzing, auditing, and designing automated decision-making systems that will be fair, accountable, and transparent. As specific goals to broaden the impact of this research, the PIs will develop a course curriculum to educate the next generation of data scientists on the ethical, legal, and societal implications of algorithmic decision-making, with the intent that they will then take this understanding into their jobs as they enter the workforce. Initial efforts by the PIs have attracted students from underrepresented groups in computer science, and they will continue these efforts. The PIs will also explore the legal and policy ramifications of this research, and develop best practice guidelines for the use of their tools by policy makers, lawyers, journalists, and other practitioners.The PIs will explore the technical subject of this project in three ways. Firstly, they will develop a sound theoretical framework for reasoning about algorithmic fairness. This framework carefully separates mechanisms, beliefs, and assumptions in order to make explicit implicitly held assumptions about the nature of fairness in learning. Secondly, by examining the entire pipeline of tasks associated with learning, they will identify hitherto unexplored areas where bias may be unintentionally introduced into learning as well as novel problems associated with ensuring fairness. These include the initial stages of data preparation, various kinds of fairness-aware learning, and evaluation. They will also investigate the problem of feedback: when actions based on a biased learned model might cause a feedback loop that changes reality and leads to more bias.

数据驱动的建模已经超越了消费者预测和建议的领域，进入了对我们日常生活产生深远影响的政策和规划领域。数据分析工具正被用于预测犯罪，选择工作候选人，识别安全威胁，确定信用风险，甚至决定患者的治疗计划和干预措施。自动化学习和挖掘工具可以处理数量惊人的各种数据，以检测模式并进行预测。正如人们迅速变得清楚的那样，这些工具也可以引入歧视性行为，并放大它们所训练的系统中的偏见。在这个项目中，PI将研究算法决策中的歧视和偏见问题。通过研究数据管道的各个方面（从数据准备到学习，评估和反馈），他们将开发用于分析，审计和设计公平，负责和透明的自动化决策系统的工具。作为扩大这项研究影响的具体目标，PI将开发一门课程，教育下一代数据科学家关于算法决策的伦理、法律的和社会影响，目的是让他们在进入劳动力市场时将这种理解融入工作中。PI的初步努力吸引了来自计算机科学代表性不足群体的学生，他们将继续这些努力。研究所还将探讨这一研究的法律的和政策影响，并为决策者、律师、记者和其他从业人员使用其工具制定最佳实践指南。首先，他们将开发一个合理的理论框架来推理算法的公平性。这个框架仔细分离机制，信念和假设，以使明确隐含的假设，在学习中的公平性的性质。其次，通过检查与学习相关的整个任务流程，他们将确定迄今为止尚未探索的领域，这些领域可能会无意中引入偏见，以及与确保公平相关的新问题。这些包括数据准备的初始阶段，各种公平意识学习和评估。他们还将研究反馈的问题：基于有偏见的学习模型的行为可能会导致一个改变现实并导致更多偏见的反馈循环。

项目成果

Fairness and Abstraction in Sociotechnical Systems

• DOI: 10.1145/3287560.3287598
• 发表时间: 2019-01
• 期刊: Proceedings of the Conference on Fairness, Accountability, and Transparency
• 作者: Andrew D. Selbst;D. Boyd;Sorelle A. Friedler;Suresh Venkatasubramanian;J. Vertesi

Decision making with limited feedback: Error bounds for predictive policing and recidivism prediction

• 发表时间: 2018
• 期刊: Engineering and Technology Journal
• 作者: D. Ensign;Sorelle A. Frielder;Scott Neville;Carlos Scheidegger;Suresh Venkatasubramanian;M. Mohri;Karthik Sridharan

Auditing Black-Box Models for Indirect Influence

审计黑盒模型的间接影响

• DOI: 10.1109/icdm.2016.0011
• 发表时间: 2016
• 期刊: IEEE 16th International Conference on Data Mining (ICDM
• 作者: Adler, Philip;Falk, Casey;Friedler, Sorelle A.;Rybeck, Gabriel;Scheidegger, Carlos;Smith, Brandon;Venkatasubramanian, Suresh
• 通讯作者: Venkatasubramanian, Suresh

Disentangling Influence: Using disentangled representations to audit model predictions

解缠结影响：使用解缠结表示来审核模型预测

• 发表时间: 2019
• 期刊: Proceedings of Neural Information Processing Systems (NeurIPS
• 作者: Marx, Charles;Phillips, Richard;Friedler, Sorelle A.;Scheidegger, Carlos;Venkatasubramanian, Suresh
• 通讯作者: Venkatasubramanian, Suresh

Fairness in representation: quantifying stereotyping as a representational harm

代表性的公平性：将刻板印象量化为代表性伤害

• 发表时间: 2019
• 期刊: Proceedings of the 2019 SIAM International Conference on Data Mining
• 作者: Abbasi, Mohsen;Friedler, Sorelle A.;Scheidegger, Carlos;Venkatasubramanian, Suresh
• 通讯作者: Venkatasubramanian, Suresh