On Privacy and Personalization in Cross-Silo Federated Learning
On Privacy and Personalization in Cross-Silo Federated Learning
复制标题
DOI：
10.48550/arxiv.2206.07902
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Ziyu Liu;Shengyuan Hu;Zhiwei Steven Wu;Virginia Smith
中科院分区：
文献类型：
--
作者：
Ziyu Liu;Shengyuan Hu;Zhiwei Steven Wu;Virginia Smith
虽然差分隐私（DP）在跨设备联邦学习（FL）中的应用已得到充分研究，但目前缺乏针对跨孤岛联邦学习中DP及其影响的相关研究。跨孤岛联邦学习的特点是客户端数量有限，而每个客户端包含众多数据主体。在跨孤岛联邦学习中，常见的客户端层面的DP概念不太适用，因为现实世界的隐私法规通常关注的是孤岛内的数据主体，而非孤岛本身。在这项研究中，我们转而考虑一种针对孤岛的样本层面DP的替代概念，即各个孤岛为其本地样本设定各自的隐私目标。在此设定下，我们重新审视了联邦学习中个性化的作用。具体而言，我们表明均值正则化多任务学习（MR - MTL）这一简单的个性化框架，是跨孤岛联邦学习的一个强大基线：在更强的隐私要求下，各孤岛有动力加强彼此之间的联合，以减轻DP噪声，相较于标准基线方法能带来持续的性能提升。我们对竞争方法进行了实证研究，并从理论上对MR - MTL在均值估计方面进行了刻画，突出了隐私与跨孤岛数据异质性之间的相互作用。我们的工作旨在为隐私保护的跨孤岛联邦学习建立基线，并确定该领域未来工作的关键方向。
While the application of differential privacy (DP) has been well-studied in cross-device federated learning (FL), there is a lack of work considering DP and its implications for cross-silo FL, a setting characterized by a limited number of clients each containing many data subjects. In cross-silo FL, usual notions of client-level DP are less suitable as real-world privacy regulations typically concern the in-silo data subjects rather than the silos themselves. In this work, we instead consider an alternative notion of silo-specific sample-level DP, where silos set their own privacy targets for their local examples. Under this setting, we reconsider the roles of personalization in federated learning. In particular, we show that mean-regularized multi-task learning (MR-MTL), a simple personalization framework, is a strong baseline for cross-silo FL: under stronger privacy requirements, silos are incentivized to federate more with each other to mitigate DP noise, resulting in consistent improvements relative to standard baseline methods. We provide an empirical study of competing methods as well as a theoretical characterization of MR-MTL for mean estimation, highlighting the interplay between privacy and cross-silo data heterogeneity. Our work serves to establish baselines for private cross-silo FL as well as identify key directions of future work in this area.