Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs
Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs
复制标题
DOI：
--
复制
发表时间：
2019-01
期刊：
影响因子：
--
通讯作者：
Valentina Zantedeschi;A. Bellet;M. Tommasi
中科院分区：
文献类型：
--
作者：
Valentina Zantedeschi;A. Bellet;M. Tommasi
我们考虑完全去中心化的机器学习场景，其中许多拥有个人数据集的用户通过本地点对点交换协作学习模型，而无需中央协调员。我们建议训练个性化模型，利用描述用户个人任务之间关系的协作图，我们与模型共同学习。我们完全分散的优化过程在以贪婪提升方式训练给定图的非线性模型和更新给定模型的协作图（具有受控稀疏性）之间交替。在整个过程中，用户仅与少数对等点（图中的直接邻居和一些随机用户）交换消息，确保该过程自然地扩展到大量用户。我们分析了我们的方法的收敛速度、内存和通信复杂性，并证明了其与合成数据集和真实数据集上的竞争技术相比的优势。
We consider the fully decentralized machine learning scenario where many users with personal datasets collaborate to learn models through local peer-to-peer exchanges , without a central coordinator. We propose to train personalized models that leverage a collaboration graph describing the relationships between the users' personal tasks, which we learn jointly with the models. Our fully decentralized optimization procedure alternates between training nonlinear models given the graph in a greedy boosting manner, and updating the collaboration graph (with controlled sparsity) given the models. Throughout the process, users exchange messages only with a small number of peers (their direct neighbors in the graph and a few random users), ensuring that the procedure naturally scales to large numbers of users. We analyze the convergence rate, memory and communication complexity of our approach, and demonstrate its benefits compared to competing techniques on synthetic and real datasets.