Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning
Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning
复制标题
DOI：
--
复制
发表时间：
2022-02
期刊：
影响因子：
--
通讯作者：
A. Bietti;Chen-Yu Wei;Miroslav Dudík;J. Langford;Zhiwei Steven Wu
中科院分区：
文献类型：
--
作者：
A. Bietti;Chen-Yu Wei;Miroslav Dudík;J. Langford;Zhiwei Steven Wu
大规模机器学习系统通常涉及分布在用户集合中的数据。联邦学习算法通过将模型更新传递到中央服务器而不是整个数据集来利用这种结构。在本文中，我们研究了随机优化算法的个性化联邦学习设置，涉及本地和全局模型的用户级（联合）差分隐私。虽然学习私有全局模型会导致隐私成本，但局部学习是完全私有的。我们提供的泛化保证表明，协调本地学习与私人集中式学习产生了普遍有用的和改进的准确性和隐私之间的权衡。我们用合成和真实世界数据集上的实验来说明我们的理论结果。
Large-scale machine learning systems often involve data distributed across a collection of users. Federated learning algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We provide generalization guarantees showing that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.