Personalized Online Federated Learning with Multiple Kernels
Personalized Online Federated Learning with Multiple Kernels
复制标题
DOI：
10.48550/arxiv.2311.05108
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
P. M. Ghari;Yanning Shen
中科院分区：
文献类型：
--
作者：
P. M. Ghari;Yanning Shen
多核学习（MKL）在在线非线性函数逼近方面表现出良好的性能。联合学习使一组学习者（称为客户端）能够在客户端之间分布的数据上训练MKL模型，以执行在线非线性函数逼近。在线联邦MKL中存在一些需要解决的挑战：i）通信效率，特别是当考虑大量内核时，ii）客户端之间的异构数据分布。本论文开发了一个算法框架，使客户端与服务器进行通信，发送他们的更新与负担得起的通信成本，而客户端采用一个大字典的内核。利用随机特征（RF）近似，本文提出了可扩展的在线联邦MKL算法。我们证明，使用所提出的在线联邦MKL算法，每个客户端享有次线性遗憾的RF近似的最佳内核在事后，这表明该算法可以有效地处理异构的数据分布在客户端之间。在真实的数据集上的实验结果表明，与其他在线联邦核学习算法相比，该算法具有明显的优势。
Multi-kernel learning (MKL) exhibits well-documented performance in online non-linear function approximation. Federated learning enables a group of learners (called clients) to train an MKL model on the data distributed among clients to perform online non-linear function approximation. There are some challenges in online federated MKL that need to be addressed: i) Communication efficiency especially when a large number of kernels are considered ii) Heterogeneous data distribution among clients. The present paper develops an algorithmic framework to enable clients to communicate with the server to send their updates with affordable communication cost while clients employ a large dictionary of kernels. Utilizing random feature (RF) approximation, the present paper proposes scalable online federated MKL algorithm. We prove that using the proposed online federated MKL algorithm, each client enjoys sub-linear regret with respect to the RF approximation of its best kernel in hindsight, which indicates that the proposed algorithm can effectively deal with heterogeneity of the data distributed among clients. Experimental results on real datasets showcase the advantages of the proposed algorithm compared with other online federated kernel learning ones.