CAREER: Enabling data valuation and deletion in human-centered machine learning

随着数据成为技术和经济发展的重要驱动力，了解数据在不同应用中的价值至关重要。该项目开发了一种计算方法，以量化当数据用于训练预测算法时，什么类型的数据或多或少有用。这种数据价值的特征很重要，因为它使用户能够过滤掉质量差的数据，并确定未来收集的重要数据。除了数据估值，该项目还开发了补充方法，以方便从预测算法中删除数据。这将允许用户快速删除质量差的数据或可能存在隐私问题的数据。数据估值和数据删除是最近政策的核心方面，旨在使个人能够控制第三方如何使用和货币化其数据。本项目中开发的方法可以为这些政策的实施提供信息。 本研究将Shapley值的概念从经济学中延伸出来，发展出一个数据评估的框架。Shapley值衡量单个组件对整个组的贡献。该项目将建立数据Shapley值的严格统计理论，以及用于估计大型数据集上Shapley值的新的可扩展算法。此外，修改数据Shapley值放宽其约束将进行调查。计算数据Shapley值涉及迭代地删除某些数据点，并测量这种删除对经训练的机器学习模型性能的影响。这一提法将数据评价与数据删除分项目密切联系起来。后者的目标是有效地从机器学习模型中删除训练数据的子集，而无需从头开始重新训练。该奖项反映了NSF的法定使命，并已被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

As data becomes an essential driver of technological and economic developments, it is critical to understand the value of data in different applications. This project develops a computational approach to quantify what type of data is more or less useful when the data is used to train prediction algorithms. This characterization of data value is important because it enables users to filter out poor quality data and to identify data that are important to collect in the future. In addition to data valuation, the project also develops complementary methods to facilitate deleting data from prediction algorithms. This would allow users to quickly remove poor quality data or data that might have privacy concerns from algorithms. Data valuation and data deletion are core aspects of recent policies aimed to enable individuals control over how their data is used and monetized by third-parties. The methods developed in this project can inform the implementation of such policies. This project develops a framework for data valuation based on extending the concept of Shapley value from economics. Shapley value measures how individual components contribute to the whole group. This project will build a rigorous statistical theory of data Shapley value, together with new scalable algorithms for estimating Shapley values on large datasets. Moreover, modifications to data Shapley value by relaxing its constraints will be investigated. Computing data Shapley value involves iteratively deleting certain data points and measuring the effect of this deletion on the performance of the trained machine learning model. This formulation closely links data valuation with the data deletion subproject. The goal of the latter is to efficiently delete subsets of the training data from a machine learning model without having to retrain from scratch. The data valuation and deletion methods will be implemented and validated on large publicly available biomedical datasets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.