Communication-Efficient Distributed Learning: An Overview
Communication-Efficient Distributed Learning: An Overview
复制标题
DOI：
10.1109/jsac.2023.3242710
复制
发表时间：
2023-04
影响因子：
16.4
通讯作者：
Xuanyu Cao;T. Başar;S. Diggavi;Y. Eldar;K. Letaief;H. Poor;Junshan Zhang
中科院分区：
文献类型：
--
作者：
Xuanyu Cao;T. Başar;S. Diggavi;Y. Eldar;K. Letaief;H. Poor;Junshan Zhang
分布式学习被设想为下一代智能网络的基石，其中智能代理（例如移动的设备、机器人和传感器）相互交换信息或与参数服务器交换信息，以协作训练机器学习模型，而无需将原始数据上传到中央实体进行集中处理。通过利用个体代理的计算/通信能力，分布式学习范式可以减轻中央处理器的负担，并有助于保护用户的数据隐私。尽管其有前途的应用，分布式学习的缺点是它需要迭代的信息交换在无线信道上，这可能会导致高通信开销在许多实际系统中，有限的无线电资源，如能量和带宽无法负担。为了克服这一通信瓶颈，迫切需要开发通信高效的分布式学习算法，能够降低通信成本，同时实现令人满意的学习/优化性能。在本文中，我们提出了一个全面的调查，目前流行的通信有效的分布式学习的方法，包括减少通信的数量，压缩和量化的交换信息，无线电资源管理有效的学习，和博弈论机制激励用户参与。我们还指出了未来研究的潜在方向，以进一步提高分布式学习在各种场景中的通信效率。
Distributed learning is envisioned as the bedrock of next-generation intelligent networks, where intelligent agents, such as mobile devices, robots, and sensors, exchange information with each other or a parameter server to train machine learning models collaboratively without uploading raw data to a central entity for centralized processing. By utilizing the computation/communication capability of individual agents, the distributed learning paradigm can mitigate the burden at central processors and help preserve data privacy of users. Despite its promising applications, a downside of distributed learning is its need for iterative information exchange over wireless channels, which may lead to high communication overhead unaffordable in many practical systems with limited radio resources such as energy and bandwidth. To overcome this communication bottleneck, there is an urgent need for the development of communication-efficient distributed learning algorithms capable of reducing the communication cost and achieving satisfactory learning/optimization performance simultaneously. In this paper, we present a comprehensive survey of prevailing methodologies for communication-efficient distributed learning, including reduction of the number of communications, compression and quantization of the exchanged information, radio resource management for efficient learning, and game-theoretic mechanisms incentivizing user participation. We also point out potential directions for future research to further enhance the communication efficiency of distributed learning in various scenarios.