CAREER: Data Preparation for Trusted and Fair Data Science

机器学习正在成为数据科学应用程序的标准选择，这些应用程序涉及各种应用领域的自动决策。经过精心设计、具有学习能力的系统有可能消除人类决策中一些不受欢迎的方面，包括有偏见的判断。然而，众所周知，这些系统会加强系统偏见和歧视，这些偏见和歧视反映在它们所训练的数据中。歧视性结果是有害的，因为它们侵犯了人权，阻碍了社会对机器学习的信任。该项目将开发新技术，以实现稳健、公平和可解释的数据驱动决策系统的潜力。为了实现这一目标，该项目以数据准备和调试技术为中心，以确保底层训练数据和数据处理过程没有意外错误。该项目将展示数据质量对于在实际领域中实现对数据驱动决策系统的信任的重要性。该项目将促进对负责任数据科学领域的理解，特别是数据质量问题和数据准备步骤如何影响下游机器学习模型和数据科学管道的公平性和偏见。这个项目的技术目标分为三个重点，并辅以中期评价计划。第一个重点是开发工具来检测机器学习模型和管道结果中的偏差原因，并建议潜在的数据修复以减轻这些偏差。第二个重点是开发评估数据的有效性或适用性的方法，以学习公平和值得信赖的机器学习模型。第三个重点是建立一个框架，使不同的人的作用和他们在减少偏见方面的专业知识都能参与进来。总之，这些技术将增强我们对数据质量和数据准备如何影响决策的理解，并将把数据作为理解和调试数据科学应用程序的不良行为的工具。该项目的研究结果将为未来设计更健壮和公平的数据科学应用程序的研究提供信息。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning is becoming the standard choice for data science applications that involve automated decision-making for a variety of application domains. Designed carefully, learning-enabled systems have the potential to eliminate some undesirable aspects of human decision-making, including biased judgments. However, these systems are known to reinforce systemic biases and discrimination reflected in the data they are trained on. Discriminatory outcomes are harmful because they violate human rights and impede societal trust in machine learning. This project will develop novel technologies to realize the potential of robust, fair, and explainable data-driven decision- making systems. Toward this goal, the project centers on data preparation and debugging techniques to ensure that the underlying training data and data handling processes are devoid of unexpected errors. The project will demonstrate the importance of data quality in enabling trust in data-driven decision-making systems in practical domains.This project will advance understanding in the field of responsible data science, particularly on how data quality issues and data preparation steps impact fairness and bias of downstream machine learning models and data science pipelines. The technical aims of this project are divided into three thrusts that are complemented by intermediate evaluation plans. The first thrust develops tools to detect the causes of bias in the outcomes of machine learning models and pipelines and suggests potential data fixes to mitigate those biases. The second thrust develops approaches to assess the validity or suitability of data for learning fair and trustworthy machine learning models. The third thrust develops a framework to involve the different human roles and their expertise for bias mitigation. Together, these techniques will enhance our understanding of how data quality and data preparation influence decision-making and will spotlight data as a tool for understanding and debugging undesired behavior of data science applications. Findings from this project will inform future research on designing more robust and fair data science applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.