Collaborative Research: MLWiNS: Distributed Learning over Multi-Access Channels: From Bandlimited Coordinate Descent to Gradient Sketching

最近机器学习和人工智能的技术进步浪潮导致了广泛的应用和公众意识。与此同时，高速无线网络服务的快速增长为未来涉及大量智能物联网设备的分布式学习提供了机会。该项目针对分布式学习系统中无线连接的有限可靠性和边缘节点的计算限制所带来的几个技术挑战。克服这些挑战对于网络边缘分布式机器学习所需的大量计算、通信和协调任务至关重要。该项目专注于在计算、功率和带宽的限制下，在无线MAC通道上开发创新的边缘学习算法，可以显著影响各种物联网应用中的无线边缘学习，从交通、安全和农业，到能源效率、电子卫生和智能基础设施。这项研究的更广泛影响还将通过为K-12、女性和代表性不足的少数民族学生提供STEM机会来提供许多教育机会。该合作项目将开发一种创新的网络架构，用于无线多接入信道上的分布式学习。具体而言，pi将采用原则性方法开发集成的无线边缘学习框架，使用基于梯度的方法和无梯度、零阶优化方面的最新进展，同时以整体方式考虑其中的计算、功率和带宽限制。所开发的方法还将扩展到无线MAC下的分布式在线学习和强化学习的设置。pi将专注于优化在带宽限制下无线网络内通信的基于梯度稀疏的通信效率的本地更新；每个发送方基于学习梯度和信道条件智能地进行传输功率分配。一个重要的目标是开发一种新的基于学习的框架，用于有效的无线信道估计和更新，以实现有效的功率控制和学习。该项目将设计对无线信道不确定性具有鲁棒性的边缘学习算法。pi团队需要全面研究无线带宽和功率约束对边缘学习算法精度和收敛速度的影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The recent wave of technological advances in machine learning and artificial intelligence has led to widespread applications and public awareness. At the same time, the rapid growth of high-speed wireless network services presents an opportunity for future distributed learning involving a vast number of smart IoT devices. This project targets several technical challenges posed by the limited reliability of wireless connections and computational constraints of the edge nodes in distributed learning systems. Overcoming these challenges is vital to the plethora of computation, communication, and coordination tasks required by distributed machine learning at the network edge. Centered on developing innovative edge learning algorithms over wireless MAC channels under the constraints of computing, power, and bandwidth, this project can significantly impact wireless edge learning in a variety of IoT applications, ranging from transportation, safety, and agriculture, to energy efficiency, e-health, and smart infrastructure. The broader impact of this research will also come through many educational opportunities by providing opportunities in STEM to K-12, women, and underrepresented minority students. This collaborative project will develop an innovative network architecture for distributed learning over wireless multi-access channels. Specifically, the PIs will take a principled approach to develop an integrated wireless edge learning framework, using both gradient-based methods and also very recent advances in gradient-free, zero-order optimization, while taking into account the constraints in computing, power and bandwidth therein, in a holistic manner. The developed methods will be also extended to the setting of distributed online learning and reinforcement learning under wireless MAC. The PIs will focus on optimizing communication-efficient gradient sparsification based local updates that are communicated within the wireless network under bandwidth constraints; and each sender intelligently carries out transmission power allocation based on learning gradient and channel conditions. One important objective is to develop a novel learning-based framework for efficient wireless channel estimation and update to enable effective power control and learning. The project will devise edge learning algorithms that are robust against wireless channel uncertainty. The team of PIs shall comprehensively investigate the impact of the wireless bandwidth and power constraint on both the accuracy and convergence speed of edge learning algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.