Federated Learning with Matched Averaging
Federated Learning with Matched Averaging
复制标题
DOI：
--
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Hongyi Wang;M. Yurochkin;Yuekai Sun;Dimitris Papailiopoulos;Y. Khazaeni
中科院分区：
文献类型：
--
作者：
Hongyi Wang;M. Yurochkin;Yuekai Sun;Dimitris Papailiopoulos;Y. Khazaeni
联合学习允许边缘设备协作学习共享模型，同时将训练数据保存在设备上，将模型训练的能力与将数据存储在云中的需求解耦。我们提出了联邦匹配平均（FedMA）算法，该算法专为现代神经网络架构（如卷积神经网络（CNN）和LSTM）的联邦学习而设计。FedMA通过匹配和平均隐藏元素（即卷积层的通道; LSTM的隐藏状态;全连接层的神经元）以分层方式构建共享全局模型。我们的实验表明，FedMA在真实的世界数据集上训练的深度CNN和LSTM架构上优于流行的最先进的联邦学习算法，同时提高了通信效率。
Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud. We propose Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural network architectures e.g. convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements (i.e. channels for convolution layers; hidden states for LSTM; neurons for fully connected layers) with similar feature extraction signatures. Our experiments indicate that FedMA outperforms popular state-of-the-art federated learning algorithms on deep CNN and LSTM architectures trained on real world datasets, while improving the communication efficiency.