Collaborative Research: MLWiNS: Distributed Learning over Multi-Access Channels: From Bandlimited Coordinate Descent to Gradient Sketching

最近机器学习和人工智能的技术进步浪潮带来了广泛的应用和公众意识。与此同时，高速无线网络服务的快速增长为涉及大量智能物联网设备的未来分布式学习提供了机会。该项目针对分布式学习系统中无线连接的有限可靠性和边缘节点的计算约束所带来的几个技术挑战。克服这些挑战对于网络边缘的分布式机器学习所需的大量计算、通信和协调任务至关重要。该项目致力于在计算、功耗和带宽的限制下通过无线MAC信道开发创新的边缘学习算法，可以显著影响各种物联网应用中的无线边缘学习，从交通、安全和农业到能源效率、电子健康和智能基础设施。这项研究的更广泛影响也将通过为K-12，妇女和代表性不足的少数民族学生提供STEM机会来获得许多教育机会。 这个合作项目将开发一个创新的网络架构，用于通过无线多路访问信道进行分布式学习。具体而言，PI将采取原则性的方法来开发集成的无线边缘学习框架，使用基于梯度的方法以及无梯度、零阶优化的最新进展，同时以整体方式考虑其中的计算、功率和带宽限制。所开发的方法也将扩展到无线MAC下的分布式在线学习和强化学习的设置。PI将专注于优化在带宽约束下在无线网络内传送的基于通信高效梯度稀疏化的本地更新;并且每个发送器基于学习梯度和信道条件智能地执行传输功率分配。一个重要的目标是开发一种新的基于学习的框架，用于有效的无线信道估计和更新，以实现有效的功率控制和学习。该项目将设计边缘学习算法，对无线信道的不确定性具有鲁棒性。PI团队将全面调查无线带宽和功率限制对边缘学习算法的准确性和收敛速度的影响。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估而被认为值得支持。

The recent wave of technological advances in machine learning and artificial intelligence has led to widespread applications and public awareness. At the same time, the rapid growth of high-speed wireless network services presents an opportunity for future distributed learning involving a vast number of smart IoT devices. This project targets several technical challenges posed by the limited reliability of wireless connections and computational constraints of the edge nodes in distributed learning systems. Overcoming these challenges is vital to the plethora of computation, communication, and coordination tasks required by distributed machine learning at the network edge. Centered on developing innovative edge learning algorithms over wireless MAC channels under the constraints of computing, power, and bandwidth, this project can significantly impact wireless edge learning in a variety of IoT applications, ranging from transportation, safety, and agriculture, to energy efficiency, e-health, and smart infrastructure. The broader impact of this research will also come through many educational opportunities by providing opportunities in STEM to K-12, women, and underrepresented minority students. This collaborative project will develop an innovative network architecture for distributed learning over wireless multi-access channels. Specifically, the PIs will take a principled approach to develop an integrated wireless edge learning framework, using both gradient-based methods and also very recent advances in gradient-free, zero-order optimization, while taking into account the constraints in computing, power and bandwidth therein, in a holistic manner. The developed methods will be also extended to the setting of distributed online learning and reinforcement learning under wireless MAC. The PIs will focus on optimizing communication-efficient gradient sparsification based local updates that are communicated within the wireless network under bandwidth constraints; and each sender intelligently carries out transmission power allocation based on learning gradient and channel conditions. One important objective is to develop a novel learning-based framework for efficient wireless channel estimation and update to enable effective power control and learning. The project will devise edge learning algorithms that are robust against wireless channel uncertainty. The team of PIs shall comprehensively investigate the impact of the wireless bandwidth and power constraint on both the accuracy and convergence speed of edge learning algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.