BIGDATA: F: Collaborative Research: Foundations of Responsible Data Management

大数据技术有望改善人们的生活，加速科学发现和创新，并带来积极的社会变革。然而，如果不负责任地使用，同样的技术可能会加剧不平等，限制问责制并侵犯个人隐私：不可复制的结果可能影响全球经济政策；搜索引擎的算法变化可以影响选举并煽动暴力；基于有偏见数据的模型可能使刑事司法系统中的歧视合法化并放大；算法招聘做法可能会默默地加剧多样性问题，并可能违反法律；违反隐私和安全可能会削弱用户的信任，并使公司面临法律和财务后果。这个项目的重点是负责任地使用大数据技术——按照伦理和道德规范，以及法律和政策的考虑。这个项目为数据管理技术建立了一个基本的新角色，在这个角色中，跨生命周期管理数据的负责任使用成为一个核心的系统需求。这个项目更广泛的目标是帮助引领数据科学的新阶段，在这个阶段，技术不仅要考虑模型的准确性，还要确保它所依赖的数据尊重相关的法律、社会规范和对人类的影响。本项目定义了负责任的数据管理的属性，包括公平性（以及代表性和多样性的相关概念）、透明度（和问责制）和数据保护。它补充了数据挖掘和机器学习社区所做的工作，这些社区的重点是分析数据分析生命周期中最后一步的公平性、问责制和透明度，并考虑了数据分析上游可能引入的问题：在数据集选择、清理、预处理、集成和共享期间。该项目开发概念框架和算法技术，在数据使用生命周期的所有阶段支持公平性、透明度和数据保护属性：从数据发现和获取开始，通过清理、集成、查询和最终分析。这些贡献有三个目的。目标1考虑负责任的数据集发现、分析和集成。目标2考虑负责任的查询处理，并为声明性规范、检查和执行公平性、代表性和多样性开发了一个通用框架。目标3将数据保护纳入生命周期，开发促进敏感数据共享的技术，并考虑隐私和透明度之间的权衡。该项目旨在围绕负责任的数据管理建立一个多学科研究议程，作为实现决策和预测系统公平、问责制和透明度的关键因素。有关该项目的更多信息，请访问DataResponsibly.com。

Big Data technology promises to improve people's lives, accelerate scientific discovery and innovation, and bring about positive societal change. Yet, if not used responsibly, this same technology can reinforce inequity, limit accountability and infringe on the privacy of individuals: irreproducible results can influence global economic policy; algorithmic changes in search engines can sway elections and incite violence; models based on biased data can legitimize and amplify discrimination in the criminal justice system; algorithmic hiring practices can silently reinforce diversity issues and potentially violate the law; privacy and security violations can erode the trust of users and expose companies to legal and financial consequences. The focus of this project is on using Big Data technology responsibly -- in accordance with ethical and moral norms, and legal and policy considerations. This project establishes a foundational new role for data management technology, in which managing the responsible use of data across the lifecycle becomes a core system requirement. The broader goal of this project is to help usher in a new phase of data science, in which the technology considers not only the accuracy of the model but also ensures that the data on which it depends respect the relevant laws, societal norms, and impacts on humans. This project defines properties of responsible data management, which include fairness (and the related concepts of representativeness and diversity), transparency (and accountability), and data protection. It complements what is done in the data mining and machine learning communities, where the focus is on analyzing fairness, accountability and transparency of the final step in the data analysis lifecycle, and considers the problems that can be introduced upstream from data analysis: during dataset selection, cleaning, pre-processing, integration, and sharing. This project develops conceptual frameworks and algorithmic techniques that support fairness, transparency and data protection properties through all stages of the data usage lifecycle: beginning with data discovery and acquisition, through cleaning, integration, querying, and ultimately analysis. The contributions are structured along three aims. Aim 1 considers responsible dataset discovery, profiling, and integration. Aim 2 considers responsible query processing and develops a general framework for declarative specification, checking and enforcement of fairness, representativeness and diversity. Aim 3 incorporates data protection into the lifecycle, develops techniques to facilitate sharing of sensitive data, and considers the tradeoffs between privacy and transparency. This project is poised to establish a multidisciplinary research agenda around responsible data management as a critical factor in enabling fairness, accountability and transparency in decision-making and prediction systems. Additional information about the project is available at DataResponsibly.com.