Decentralized Federated Learning Over Slotted ALOHA Wireless Mesh Networking
Decentralized Federated Learning Over Slotted ALOHA Wireless Mesh Networking
复制标题
DOI：
10.1109/access.2023.3246924
复制
发表时间：
2023
期刊：
影响因子：
3.9
通讯作者：
Abdelaziz Salama;Achilleas Stergioulis;A. Hayajneh;Syed Ali Raza Zaidi;D. McLernon;Ian Robertson
中科院分区：
文献类型：
--
作者：
Abdelaziz Salama;Achilleas Stergioulis;A. Hayajneh;Syed Ali Raza Zaidi;D. McLernon;Ian Robertson
联邦学习（FL）提供了一种机制，允许对机器学习（ML）模型进行分散式训练，从而实现隐私保护。经典的FL被实现为客户端-服务器系统，这被称为集中式联邦学习（CFL）。CFL存在固有的挑战，因为所有参与者都需要与中央服务器交互，从而导致潜在的通信瓶颈和单点故障。此外，由于实施成本和复杂性，在某些场景中很难拥有中央服务器。本研究旨在通过一跳邻居使用分散式联邦学习（DFL），而无需中央服务器。这种协作取决于通信网络的动态，例如，网络的拓扑结构、MAC协议以及链路上的大规模和小规模衰落。在本文中，我们采用随机几何模型明确这些动态，使我们能够量化的DFL的性能。核心目标是在不牺牲隐私的情况下实现更好的分类，同时适应网络动态。在本文中，我们感兴趣的是这些拓扑在实际部署时如何影响ML的性能。所提出的系统是在一个众所周知的MINST数据集上训练的，该数据集包含60 K图像的标记数据样本，每个图像的大小为28 × 28像素，并且为每个参与者的设备分配1000个该MNIST数据集的随机样本。参与者的设备实现CNN模型作为分类器模型。为了评估模型的性能，从网络中随机选择一些参与者。由于通信过程中的随机性，这些参与者与邻居中随机数量的节点交互以交换模型参数，这些参数随后用于更新参与者的个体模型。这些参与者与不同数量的邻居成功连接，以交换参数并更新其全球模型。结果表明，在训练设置中使用三种不同的模型优化器（即，SGD、ADAM和RMSprop优化器）。因此，网状网络上的DFL在物联网系统中表现出更大的灵活性，这降低了通信成本并提高了收敛速度，可以优于CFL。
Federated Learning (FL) presents a mechanism to allow decentralized training for machine learning (ML) models inherently enabling privacy preservation. The classical FL is implemented as a client-server system, which is known as Centralised Federated Learning (CFL). There are challenges inherent in CFL since all participants need to interact with a central server resulting in a potential communication bottleneck and a single point of failure. In addition, it is difficult to have a central server in some scenarios due to the implementation cost and complexity. This study aims to use Decentralized Federated learning (DFL) without a central server through one-hop neighbours. Such collaboration depends on the dynamics of communication networks, e.g., the topology of the network, the MAC protocol, and both large-scale and small-scale fading on links. In this paper, we employ stochastic geometry to model these dynamics explicitly, allowing us to quantify the performance of the DFL. The core objective is to achieve better classification without sacrificing privacy while accommodating for networking dynamics. In this paper, we are interested in how such topologies impact the performance of ML when deployed in practice. The proposed system is trained on a well-known MINST dataset for benchmarking, which contains labelled data samples of 60K images each with a size $28\times 28$ pixels, and 1000 random samples of this MNIST dataset are assigned for each participant’ device. The participants’ devices implement a CNN model as a classifier model. To evaluate the performance of the model, a number of participants are randomly selected from the network. Due to randomness in the communication process, these participants interact with the random number of nodes in the neighbourhood to exchange model parameters which are subsequently used to update the participants’ individual models. These participants connected successfully with a varying number of neighbours to exchange parameters and update their global models. The results show that the classification prediction system was able to achieve higher than 95% accuracy using the three different model optimizers in the training settings (i.e., SGD, ADAM, and RMSprop optimizers). Consequently, the DFL over mesh networking shows more flexibility in IoT systems, which reduces the communication cost and increases the convergence speed which can outperform CFL.