Privacy-Preserving Federated Multi-Task Linear Regression: A One-Shot Linear Mixing Approach Inspired By Graph Regularization
Privacy-Preserving Federated Multi-Task Linear Regression: A One-Shot Linear Mixing Approach Inspired By Graph Regularization
复制标题
DOI：
10.1109/icassp43922.2022.9746007
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Harlin Lee;A. Bertozzi;J. Kovacevic;Yuejie Chi
中科院分区：
文献类型：
--
作者：
Harlin Lee;A. Bertozzi;J. Kovacevic;Yuejie Chi
我们研究了多任务学习（MTL），其中多个学习任务是联合执行的，而不是单独执行的，以利用它们的相似性并提高性能。我们专注于联合多任务线性回归设置，其中每台机器都拥有自己的各个任务数据，并且禁止在机器之间共享完整的本地数据。在图正则化的推动下，我们提出了一种新颖的融合框架，仅需要局部估计的一次性通信。我们的方法线性组合局部估计，以产生每个任务的改进估计，并且我们表明，融合的理想混合权重是任务相似性和任务难度的函数。开发并证明了一种实用的算法可以显着减少合成数据的均方误差 (MSE)，并提高收入预测任务的性能，其中现实世界数据按种族分类。
We investigate multi-task learning (MTL), where multiple learning tasks are performed jointly rather than separately to leverage their similarities and improve performance. We focus on the federated multi-task linear regression setting, where each machine possesses its own data for individual tasks and sharing the full local data between machines is prohibited. Motivated by graph regularization, we propose a novel fusion framework that only requires a one-shot communication of local estimates. Our method linearly combines the local estimates to produce an improved estimate for each task, and we show that the ideal mixing weight for fusion is a function of task similarity and task difficulty. A practical algorithm is developed and shown to significantly reduce mean squared error (MSE) on synthetic data, as well as improve performance on an income prediction task where the real-world data is disaggregated by race.