Private Multi-Task Learning: Formulation and Applications to Federated Learning
Private Multi-Task Learning: Formulation and Applications to Federated Learning
复制标题
DOI：
--
复制
发表时间：
2021-08
期刊：
影响因子：
--
通讯作者：
Shengyuan Hu;Zhiwei Steven Wu;Virginia Smith
中科院分区：
文献类型：
--
作者：
Shengyuan Hu;Zhiwei Steven Wu;Virginia Smith
机器学习中的许多问题依赖于多任务学习（MTL），其目标是同时解决多个相关的机器学习任务。多任务学习在医疗、金融和物联网计算等对隐私敏感的应用领域尤为重要，在这些领域中，来自多个不同来源的敏感数据会为了学习目的而被共享。在这项研究中，我们通过联合差分隐私（JDP）对多任务学习中客户端层面的隐私概念进行了形式化定义，JDP是对差分隐私在机制设计和分布式优化方面的一种放宽。随后，我们针对均值正则化多任务学习提出了一种算法，这是个性化联邦学习应用中常用的一个目标，该算法满足联合差分隐私。我们对目标和求解器进行了分析，在隐私和效用方面都给出了可验证的保证。通过实证研究，我们发现，相对于常见联邦学习基准测试中的全局基线，我们的方法在隐私/效用权衡方面有了改进。
Many problems in machine learning rely on multi-task learning (MTL), in which the goal is to solve multiple related machine learning tasks simultaneously. MTL is particularly relevant for privacy-sensitive applications in areas such as healthcare, finance, and IoT computing, where sensitive data from multiple, varied sources are shared for the purpose of learning. In this work, we formalize notions of client-level privacy for MTL via joint differential privacy (JDP), a relaxation of differential privacy for mechanism design and distributed optimization. We then propose an algorithm for mean-regularized MTL, an objective commonly used for applications in personalized federated learning, subject to JDP. We analyze our objective and solver, providing certifiable guarantees on both privacy and utility. Empirically, we find that our method provides improved privacy/utility trade-offs relative to global baselines across common federated learning benchmarks.