CB-DSL: Communication-Efficient and Byzantine-Robust Distributed Swarm Learning on Non-i.i.d. Data
CB-DSL: Communication-Efficient and Byzantine-Robust Distributed Swarm Learning on Non-i.i.d. Data
复制标题
DOI：
10.1109/tccn.2023.3312345
复制
发表时间：
2022-08
影响因子：
8.6
通讯作者：
Xin Fan;Yue Wang;Yan Huo;Zhi Tian
中科院分区：
文献类型：
--
作者：
Xin Fan;Yue Wang;Yan Huo;Zhi Tian
物联网设备收集的有价值的数据以及机器学习（ML）的复兴刺激了边缘人工智能（AI）的最新趋势。然而，传统ML和最近的联邦学习（FL）面临着主要挑战，包括通信瓶颈，数据异构性和边缘物联网的安全问题。与此同时，物联网系统的群体性质被大多数现有文献所忽视，这需要新的分布式学习算法设计。受群居生物的生物智能（BI）成功的启发，我们提出了一种用于群物联网的新型边缘学习方法，称为通信高效和拜占庭鲁棒分布式群学习（CB-DSL），通过AI支持的随机梯度下降和BI支持的粒子群优化的整体集成。去处理那些没有身份证的人。针对数据问题和拜占庭攻击，CB-DSL引入少量的全局数据样本，并在物联网工作者之间共享，有效地消除了本地数据的异构性，充分利用了群体智能的探索-利用机制。我们的收敛性分析理论上表明，CB-DSL是上级优于标准FL具有更好的收敛行为。我们还评估了CB-DSL的模型分歧，推导其上限，这衡量了全球共享数据集的引入的有效性。
The valuable data collected by IoT devices together with the resurgence of machine learning (ML) stimulate the latest trend of artificial intelligence (AI) at the edge. However, traditional ML and recent federated learning (FL) face major challenges including communication bottleneck, data heterogeneity, and security concerns in edge IoT. Meanwhile, the swarm nature of IoT systems is overlooked by most existing literature, which calls for new designs of distributed learning algorithms. Inspired by the success of biological intelligence (BI) of gregarious organisms, we propose a novel edge learning approach for swarm IoT, called communication-efficient and Byzantine-robust distributed swarm learning (CB-DSL), through a holistic integration of AI-enabled stochastic gradient descent and BI-enabled particle swarm optimization. To deal with the non-i.i.d. data issues and Byzantine attacks, a small amount of global data samples are introduced in CB-DSL and shared among IoT workers, which alleviates the local data heterogeneity effectively and enables to fully utilize the exploration-exploitation mechanism of swarm intelligence. Our convergence analysis theoretically demonstrates that the CB-DSL is superior to the standard FL with better convergence behavior. We also evaluate the model divergence of CB-DSL by deriving its upper bound, which measures the effectiveness of the introduction of the globally shared dataset.