Rate-Adapted Decentralized Learning Over Wireless Networks
Rate-Adapted Decentralized Learning Over Wireless Networks
复制标题
通过无线网络进行速率自适应的分散学习
DOI：
10.1109/tccn.2021.3074908
复制
发表时间：
2021
影响因子：
8.6
通讯作者：
Koya Sato and Daisuke Sugimura
中科院分区：
文献类型：
--
作者：
Ryuji Kuse;Takeshi Fukusako;Akira Matsushima;青山真大，野口季彦;碓井璃菜・松本瀬名・七森公碩;Koya Sato and Daisuke Sugimura
提出了一种在无线系统中采用自适应调制和编码能力的分散学习通信策略。该工作的主要目的是解决无线系统中基于协作随机梯度下降(C-SGD)的分布式学习中的一个关键问题：传输速率和网络密度之间的关系影响学习的运行时性能。我们首先提出，密集的网络拓扑并不一定比稀疏的网络拓扑更有利于学习的迭代性能。然而，它往往会降低运行时性能，因为密集的网络拓扑需要低速率传输。在此基础上，提出了一种通信策略，在网络密度的约束下，每个节点优化其传输速率以最小化C-SGD期间的通信时间。我们对独立分布和同分布(I.I.D.)下的图像分类任务进行了数值模拟。也没有身份证明。设置。仿真结果表明，网络密度的优选设置取决于信道条件和训练样本中的偏差。此外，对自动调制分类任务的数值模拟表明，即使训练任务不同，优选设置也几乎相同。
This paper proposes a communication strategy for decentralized learning in wireless systems that employs adaptive modulation and coding capability. The main objective of this work is to address a critical issue in decentralized learning based on the cooperative stochastic gradient descent (C-SGD) over wireless systems: the relationship between the transmission rate and the network density influences the runtime performance of learning. We first present that a dense network topology does not necessarily benefit the iteration performance of learning than a sparse one. However, it tends to degrade the runtime performance because the dense network topology requires a low-rate transmission. Based on these findings, a communication strategy is proposed in which each node optimizes its transmission rate to minimize communication time during the C-SGD under the constraints of network density. We perform numerical simulations of an image classification task under both independent and identically distributed (i.i.d.) and non-i.i.d. settings. The simulation results reveal that the preferred setting for the network density depends on the channel conditions and the biases in the training samples. Furthermore, numerical simulations of an automatic modulation classification task indicate that the preferred setting is almost the same even if the training task is different.