Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams
Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
S. Denisov;H. B. McMahan;J. Rush;Adam D. Smith;Abhradeep Thakurta
中科院分区：
文献类型：
--
作者：
S. Denisov;H. B. McMahan;J. Rush;Adam D. Smith;Abhradeep Thakurta
最近的应用程序需要差分隐私自适应流的动机，我们调查的问题，在这种设置中的矩阵机制的最佳实例。我们证明了基本的理论结果的适用性矩阵分解自适应流，并提供了一个无参数的不动点算法计算最佳因子分解。我们将这个框架实例化到机器学习中自然产生的具体矩阵中，并使用由此产生的最佳机制训练用户级差分隐私模型，从而在具有用户级差分隐私的联邦学习中的一个显著问题上取得了显着改进。
Motivated by recent applications requiring differential privacy over adaptive streams, we investigate the question of optimal instantiations of the matrix mechanism in this setting. We prove fundamental theoretical results on the applicability of matrix factorizations to adaptive streams, and provide a parameter-free fixed-point algorithm for computing optimal factorizations. We instantiate this framework with respect to concrete matrices which arise naturally in machine learning, and train user-level differentially private models with the resulting optimal mechanisms, yielding significant improvements in a notable problem in federated learning with user-level differential privacy.