Retiring Adult: New Datasets for Fair Machine Learning
Retiring Adult: New Datasets for Fair Machine Learning
复制标题
DOI：
--
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Frances Ding;Moritz Hardt;John Miller;Ludwig Schmidt
中科院分区：
文献类型：
--
作者：
Frances Ding;Moritz Hardt;John Miller;Ludwig Schmidt
虽然公平社区已经认识到数据的重要性，但当涉及到表格数据时，该领域的研究人员主要依赖UCI Adult。该数据集来自1994年美国人口普查调查，已出现在数百篇研究论文中，并作为许多算法公平干预措施的开发和比较的基础。我们从可用的美国人口普查来源重建UCI成人数据的超集，并揭示限制其外部有效性的UCI成人数据集的特质。我们的主要贡献是一套来自美国人口普查调查的新数据集，这些数据集扩展了公平机器学习研究的现有数据生态系统。我们创建与收入、就业、健康、交通和住房相关的预测任务。这些数据跨越多年，涵盖美国所有州，使研究人员能够研究时间变化和地理变化。基于我们的新数据集，我们强调了与公平标准、算法干预的表现和分布转移的作用之间的权衡有关的广泛的初步经验见解。我们的研究结果为正在进行的辩论提供了信息，挑战了一些现有的叙述，并指出了未来的研究方向。我们的数据集可在https://github.com/zykls/folktables上获得。
Although the fairness community has recognized the importance of data, researchers in the area primarily rely on UCI Adult when it comes to tabular data. Derived from a 1994 US Census survey, this dataset has appeared in hundreds of research papers where it served as the basis for the development and comparison of many algorithmic fairness interventions. We reconstruct a superset of the UCI Adult data from available US Census sources and reveal idiosyncrasies of the UCI Adult dataset that limit its external validity. Our primary contribution is a suite of new datasets derived from US Census surveys that extend the existing data ecosystem for research on fair machine learning. We create prediction tasks relating to income, employment, health, transportation, and housing. The data span multiple years and all states of the United States, allowing researchers to study temporal shift and geographic variation. We highlight a broad initial sweep of new empirical insights relating to trade-offs between fairness criteria, performance of algorithmic interventions, and the role of distribution shift based on our new datasets. Our findings inform ongoing debates, challenge some existing narratives, and point to future research directions. Our datasets are available at https://github.com/zykls/folktables.