Algorithmic Watchdog for Differential Privacy: From Theory to Practice

凭借机器学习（ML）的强大进步，对手从个人不断扩大的数字足迹中收集个人信息的能力超过了任何人保持信息隐私的能力。虽然收集的数据可以通过基于ML的技术为消费者带来巨大的好处，但这种好处必须得到有意义的隐私保证。在ML中对这种保证进行推理的事实标准是差分隐私（DP）。然而，尽管它在政府和公司中被广泛采用，却没有评估和监测DP技术的标准化方法。不正确指定的DP参数可能会导致过高的效用损失，并且在更糟糕的情况下，无法提供针对攻击者的隐私。今天，这些潜在的危害是无声的：没有自动的方法来调优和监视DP滥用的算法。本研究通过创建精确描述现有DP算法中私有信息泄露风险和降低效用的数学方法来填补这一空白。这些方法将构成可扩展的“算法监管机构”的严格蓝图，以监控DP技术的误用和意外伤害。算法监管机构将减少在使用个人敏感数据（如加拿大人口普查记录）的应用程序中部署DP的潜在危害。此外，它们将从根本上影响政府（例如加拿大统计局）和公司如何实施DP，并帮助开发人员优化私人学习算法的参数，监控DP技术的性能损失，并警告潜在的滥用。拟议的工作有两个关键的新颖之处。首先，我们建议利用信息论的强大工具，共同表征现有DP算法的“操作隐私”和效用保证。必须同时跟踪这两个方面，以避免在DP部署中滥用。信息论方法的关键优势在于，它将以一种独立于算法的方式，从数学上描绘出私人学习的基本限制。这些限制反过来又作为设计和基准测试实用算法的严格蓝图。其次，该研究跨越了DP开发堆栈：它防止了从DP的（许多）数学定义到使用开源DP软件的技术实际部署的误用。该计划的总体长期目标是为政府和企业的数据科学家提供一套理论基础和算法工具，以确保有意义和可操作的隐私，并将可证明的意外伤害降到最低。这项研究将帮助政府的数据科学家了解操作隐私如何影响机器学习任务的准确性，以及如何在两种流行的DP应用程序中最佳地选择学习算法的参数：查询统计数据库和训练机器学习模型。

Armed with powerful advances in machine learning (ML), the ability of an adversary to gather personal information from an individual's expanding digital footprint is outstripping anyone's capability to keep their information private. While the data collected can have tremendous benefit for consumers via technologies built on ML, this benefit must be tempered with meaningful assurances of privacy. The de-facto standard for reasoning about such assurances in ML is differential privacy (DP). However, despite its widespread adoption in governments and corporations, there is no standardized approach for evaluating and monitoring DP technologies. Incorrectly specified DP parameters may incur a prohibitively high utility loss and-at worse-not provide privacy against an adversary. Today, these potential harms are silent: there are no automated methods for tuning and monitoring algorithms for DP misuse. This research fills this gap by creating mathematical methods that precisely characterize the risks of private information leakage and reduced utility in existing DP algorithms. These methods will constitute a rigorous blueprint for scalable "algorithmic watchdogs" that monitor DP technologies for misuse and unintended harm. Algorithmic watchdogs will reduce the potential harm of deploying DP in applications that use individual-level sensitive data such as Canada's census records. Moreover, they will fundamentally impact how DP is implemented by governments (e.g., Statistics Canada) and companies, and help developers optimally tune the parameters of private learning algorithms, monitor DP technologies for performance loss, and alert for potential misuse. The proposed work has two key novelties. First, we propose to jointly characterize the "operational privacy" and utility guarantees of existing DP algorithms by applying powerful tools from information theory. These two aspects must be simultaneously tracked to avoid misuse in DP deployments. The key advantage of the information-theoretic approach is that it will mathematically delineate the fundamental limits of private learning in an algorithmic-independent manner. These limits, in turn, serve as a rigorous blueprint for designing and benchmarking practical algorithms. Second, the research cuts across the DP development stack: it prevents misuse starting from the (many) mathematical definitions of DP to the actual deployment of the technology using open-source DP software. The overall long-term goal of this initiative is to provide data scientists in governments and corporations with a set of theoretically-grounded and algorithmic tools to guarantee meaningful and operational privacy with provably minimal unintended harms. This research will help government's data scientists understand how operational privacy can impact the accuracy of ML tasks, as well as how to optimally select parameters of learning algorithms in two popular DP applications: queries to statistical databases and training ML models.