权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

AI-DCL: EAGER: Human-in-the-Loop Fairness Optimization in Machine Learning with Minimax Loss and an Abstain Option

AI-DCL：EAGER：具有最小最大损失和弃权选项的机器学习中的人机循环公平性优化

基本信息

批准号：
1927564
负责人：
Fuxin Li
金额：
$ 30万
依托单位：
Oregon State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-09-15 至 2023-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1927564&HistoricalAwards=false
关键词：
AI DCL EAGER Human Loop

项目摘要

This project will implement machine learning algorithms that explores tradeoffs involving three factors: accuracy, fairness, and input data coverage. On the computational side, the project has the potential to bring about a paradigm shift for fair machine learning algorithms while improving uncertainty estimates for those algorithms. The project also includes the development of a human-in-the-loop optimization algorithm to enable humans to dynamically tune the three factors and thereby interact with the algorithm. On the sociological side, this work will provide a cross-cultural study of subject preferences when presented with quantifiable tradeoffs between the three specified factors; it will focus on behavioral differences between individuals in response to their interactive explorations of how the tradeoffs work. The results of this study will serve to complement prior related research that is more qualitative. The results of this study could have substantial impacts on policy making on machine learning algorithms. They could also serve to improve the public's in machine learning algorithms and enable more human-machine teams, which is important in the current era where machine learning algorithms have increasingly become black boxes while being more broadly deployed in many crucial real-life applications.The goal of this project is to study the trade-off between accuracy, fairness, and data coverage in machine learning algorithms. The research team plans to develop novel hybrid human/machine-learning algorithms with an integrated, optimizable fairness component. The specific objectives are to develop algorithms that are designed to trade-off between three factors: (1) the traditional average error objective that pertains to utility, (2) a minimax error objective that minimize the maximal error occurring to any training example, which pertains to fairness, and (3) the coverage of the algorithm on the input distribution by providing an abstain option that the algorithm can utilize when it is not confident in giving a correct answer. The team will develop their algorithms using saddle point optimization approaches in zero-sum games. A human-in-the-loop optimization algorithm will be designed for humans to dynamically tune the three factors to facilitate interaction with the algorithm. The team will use a hybrid iterative approach to algorithm design and testing that is based in grounded theory, which is widely used in the human and social sciences. Humans will provide directions (more fairness) and specify the groups they want to cover, while the real-valued changes will be automatically computed. In order to better understand of trade-offs on utility, fairness and coverage from a sociological perspective, a broad cross-cultural evaluation will be performed with multiple social-cultural groups in the United States as well as an online platform to reach global users in China and Brazil.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目将实现机器学习算法，探索涉及三个因素的权衡：准确性，公平性和输入数据覆盖率。在计算方面，该项目有可能为公平的机器学习算法带来范式转变，同时改善这些算法的不确定性估计。该项目还包括开发一种人在回路优化算法，使人类能够动态调整这三个因素，从而与算法进行交互。在社会学方面，这项工作将提供一个跨文化的主题偏好的研究时，提出了可量化的权衡之间的三个指定的因素，它将集中在个人之间的行为差异，以响应他们的互动探索如何权衡工作。这项研究的结果将有助于补充以前的相关研究，这是更定性。这项研究的结果可能会对机器学习算法的政策制定产生重大影响。在机器学习算法越来越多地成为黑箱，同时被广泛应用于许多关键的现实生活应用的当今时代，这一点非常重要。本项目的目标是研究机器学习算法中准确性、公平性和数据覆盖率之间的权衡。该研究小组计划开发新的混合人类/机器学习算法，其中包含集成的、可优化的公平组件。具体目标是开发旨在权衡以下三个因素的算法：（1）属于效用的传统平均误差目标，（2）最小化任何训练样本发生的最大误差的最小化误差目标，其属于公平性，以及（3）通过提供弃权选项来提高算法对输入分布的覆盖率，当算法不确信给出正确答案时可以利用该弃权选项。该团队将在零和游戏中使用鞍点优化方法开发他们的算法。将设计一种人在回路优化算法，供人类动态调整这三个因素，以促进与算法的交互。该团队将使用基于扎根理论的混合迭代方法来进行算法设计和测试，该方法广泛应用于人类和社会科学。人类将提供方向（更公平），并指定他们想要覆盖的群体，而实际值的变化将自动计算。为了从社会学的角度更好地理解效用、公平和覆盖面的权衡，一个广泛的跨文化评估将与多个社会-该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响力进行评估，被认为值得支持审查标准。