Algorithmic Watchdog for Differential Privacy: From Theory to Practice

点击翻译按钮获取中文摘要

Armed with powerful advances in machine learning (ML), the ability of an adversary to gather personal information from an individual's expanding digital footprint is outstripping anyone's capability to keep their information private. While the data collected can have tremendous benefit for consumers via technologies built on ML, this benefit must be tempered with meaningful assurances of privacy. The de-facto standard for reasoning about such assurances in ML is differential privacy (DP). However, despite its widespread adoption in governments and corporations, there is no standardized approach for evaluating and monitoring DP technologies. Incorrectly specified DP parameters may incur a prohibitively high utility loss and-at worse-not provide privacy against an adversary. Today, these potential harms are silent: there are no automated methods for tuning and monitoring algorithms for DP misuse. This research fills this gap by creating mathematical methods that precisely characterize the risks of private information leakage and reduced utility in existing DP algorithms. These methods will constitute a rigorous blueprint for scalable "algorithmic watchdogs" that monitor DP technologies for misuse and unintended harm. Algorithmic watchdogs will reduce the potential harm of deploying DP in applications that use individual-level sensitive data such as Canada's census records. Moreover, they will fundamentally impact how DP is implemented by governments (e.g., Statistics Canada) and companies, and help developers optimally tune the parameters of private learning algorithms, monitor DP technologies for performance loss, and alert for potential misuse. The proposed work has two key novelties. First, we propose to jointly characterize the "operational privacy" and utility guarantees of existing DP algorithms by applying powerful tools from information theory. These two aspects must be simultaneously tracked to avoid misuse in DP deployments. The key advantage of the information-theoretic approach is that it will mathematically delineate the fundamental limits of private learning in an algorithmic-independent manner. These limits, in turn, serve as a rigorous blueprint for designing and benchmarking practical algorithms. Second, the research cuts across the DP development stack: it prevents misuse starting from the (many) mathematical definitions of DP to the actual deployment of the technology using open-source DP software. The overall long-term goal of this initiative is to provide data scientists in governments and corporations with a set of theoretically-grounded and algorithmic tools to guarantee meaningful and operational privacy with provably minimal unintended harms. This research will help government's data scientists understand how operational privacy can impact the accuracy of ML tasks, as well as how to optimally select parameters of learning algorithms in two popular DP applications: queries to statistical databases and training ML models.