Collaborative Research: MLWiNS: Distributed Learning over Multi-Access Channels: From Bandlimited Coordinate Descent to Gradient Sketching

最近机器学习和人工智能领域的技术进步浪潮导致了广泛的应用和公众意识。与此同时，高速无线网络服务的快速增长为未来涉及大量智能物联网设备的分布式学习提供了机会。该项目针对分布式学习系统中无线连接的有限可靠性和边缘节点的计算限制所带来的几个技术挑战。克服这些挑战对于网络边缘的分布式机器学习所需的过多计算、通信和协调任务至关重要。该项目以开发无线MAC信道上的创新边缘学习算法为中心，在计算、功率和带宽的限制下，该项目可以显著影响从交通、安全和农业到能效、电子健康和智能基础设施等各种物联网应用中的无线边缘学习。这项研究的更广泛影响还将通过为K-12、女性和代表性不足的少数族裔学生提供STEM机会而产生许多教育机会。这一合作项目将为无线多路访问渠道上的分布式学习开发一种创新的网络架构。具体地说，私人投资机构将采取原则性的方法，利用基于梯度的方法以及在无梯度、零阶优化方面的最新进展，制定一个综合的无线边缘学习框架，同时以整体的方式考虑其中在计算、功率和带宽方面的限制。所开发的方法还将扩展到无线MAC下的分布式在线学习和强化学习的设置。PI将专注于优化通信效率高的基于梯度稀疏化的本地更新，这些本地更新在带宽限制下在无线网络内进行通信；每个发送器基于学习梯度和信道条件智能地进行传输功率分配。一个重要的目标是开发一种新的基于学习的框架，用于有效的无线信道估计和更新，以实现有效的功率控制和学习。该项目将设计出对无线信道不确定性具有健壮性的边缘学习算法。PIS团队应全面调查无线带宽和功率限制对边缘学习算法的准确性和收敛速度的影响。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The recent wave of technological advances in machine learning and artificial intelligence has led to widespread applications and public awareness. At the same time, the rapid growth of high-speed wireless network services presents an opportunity for future distributed learning involving a vast number of smart IoT devices. This project targets several technical challenges posed by the limited reliability of wireless connections and computational constraints of the edge nodes in distributed learning systems. Overcoming these challenges is vital to the plethora of computation, communication, and coordination tasks required by distributed machine learning at the network edge. Centered on developing innovative edge learning algorithms over wireless MAC channels under the constraints of computing, power, and bandwidth, this project can significantly impact wireless edge learning in a variety of IoT applications, ranging from transportation, safety, and agriculture, to energy efficiency, e-health, and smart infrastructure. The broader impact of this research will also come through many educational opportunities by providing opportunities in STEM to K-12, women, and underrepresented minority students. This collaborative project will develop an innovative network architecture for distributed learning over wireless multi-access channels. Specifically, the PIs will take a principled approach to develop an integrated wireless edge learning framework, using both gradient-based methods and also very recent advances in gradient-free, zero-order optimization, while taking into account the constraints in computing, power and bandwidth therein, in a holistic manner. The developed methods will be also extended to the setting of distributed online learning and reinforcement learning under wireless MAC. The PIs will focus on optimizing communication-efficient gradient sparsification based local updates that are communicated within the wireless network under bandwidth constraints; and each sender intelligently carries out transmission power allocation based on learning gradient and channel conditions. One important objective is to develop a novel learning-based framework for efficient wireless channel estimation and update to enable effective power control and learning. The project will devise edge learning algorithms that are robust against wireless channel uncertainty. The team of PIs shall comprehensively investigate the impact of the wireless bandwidth and power constraint on both the accuracy and convergence speed of edge learning algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.