Collaborative Research: MLWiNS: Distributed Learning over Multi-Access Channels: From Bandlimited Coordinate Descent to Gradient Sketching

协作研究：MLWiNS：多访问通道上的分布式学习：从带限坐标下降到梯度草图

• 批准号: 2003111
• 负责人: Na Li
• 金额: $ 20万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2003111&HistoricalAwards=false
• 关键词: Collaborative; Research; MLWiNS; Distributed; Learning

The recent wave of technological advances in machine learning and artificial intelligence has led to widespread applications and public awareness. At the same time, the rapid growth of high-speed wireless network services presents an opportunity for future distributed learning involving a vast number of smart IoT devices. This project targets several technical challenges posed by the limited reliability of wireless connections and computational constraints of the edge nodes in distributed learning systems. Overcoming these challenges is vital to the plethora of computation, communication, and coordination tasks required by distributed machine learning at the network edge. Centered on developing innovative edge learning algorithms over wireless MAC channels under the constraints of computing, power, and bandwidth, this project can significantly impact wireless edge learning in a variety of IoT applications, ranging from transportation, safety, and agriculture, to energy efficiency, e-health, and smart infrastructure. The broader impact of this research will also come through many educational opportunities by providing opportunities in STEM to K-12, women, and underrepresented minority students. This collaborative project will develop an innovative network architecture for distributed learning over wireless multi-access channels. Specifically, the PIs will take a principled approach to develop an integrated wireless edge learning framework, using both gradient-based methods and also very recent advances in gradient-free, zero-order optimization, while taking into account the constraints in computing, power and bandwidth therein, in a holistic manner. The developed methods will be also extended to the setting of distributed online learning and reinforcement learning under wireless MAC. The PIs will focus on optimizing communication-efficient gradient sparsification based local updates that are communicated within the wireless network under bandwidth constraints; and each sender intelligently carries out transmission power allocation based on learning gradient and channel conditions. One important objective is to develop a novel learning-based framework for efficient wireless channel estimation and update to enable effective power control and learning. The project will devise edge learning algorithms that are robust against wireless channel uncertainty. The team of PIs shall comprehensively investigate the impact of the wireless bandwidth and power constraint on both the accuracy and convergence speed of edge learning algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习和人工智能方面的技术进步浪潮导致了广泛的应用和公众意识。同时，高速无线网络服务的快速增长为将来的分布式学习提供了机会，涉及大量智能IoT设备。该项目针对的是，由无线连接的有限可靠性和分布式学习系统中边缘节点的计算限制的有限可靠性所带来的几个技术挑战。克服这些挑战对于网络边缘分布式机器学习所需的大量计算，通信和协调任务至关重要。该项目围绕计算，功率和带宽的限制，在无线MAC通道上开发创新的边缘学习算法，该项目可能会严重影响各种物联网应用中的无线边缘学习，从运输，安全性和农业，到能源效率，电子卫生和智能基础结构。这项研究的更广泛的影响还将通过许多教育机会，通过向K-12，妇女和代表性不足的少数族裔学生提供机会。 这个协作项目将开发一种创新的网络体系结构，用于通过无线多访问渠道分布式学习。具体来说，PI将采用一种原则性的方法来开发一个基于梯度的方法，以及在无梯度的零级优化方面的最新进展，同时考虑到计算，功率和带宽的约束，并以整体方式考虑了限制。开发的方法还将扩展到无线MAC下的分布式在线学习和增强学习的设置。 PI将着重于优化基于带宽约束的无线网络中传达的基于无线网络中的沟通梯度稀疏的本地更新；每个发件人都根据学习梯度和渠道条件智能地进行传输功率分配。一个重要的目标是开发一个基于学习的新型框架，以进行有效的无线渠道估计并进行更新，以实现有效的功率控制和学习。该项目将设计边缘学习算法，这些算法可抵抗无线通道不确定性。 PIS团队应全面研究无线带宽和功率约束对边缘学习算法的准确性和收敛速度的影响。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛影响的审查标准通过评估来获得支持的。

项目成果

Communication-Efficient Distributed SGD With Compressed Sensing

• DOI: 10.1109/lcsys.2021.3137859
• 发表时间: 2021-12
• 期刊: IEEE Control Systems Letters
• 影响因子: 3
• 作者: Yujie Tang;V. Ramanathan;Junshan Zhang;N. Li

Improve Single-Point Zeroth-Order Optimization Using High-Pass and Low-Pass Filters

• 发表时间: 2021-11
• 期刊: ArXiv
• 作者: Xin Chen;Yujie Tang;N. Li

Source Seeking by Dynamic Source Location Estimation

通过动态源位置估计来寻找源

• DOI: 10.1109/iros51168.2021.9636841
• 发表时间: 2021
• 期刊: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Zhang, Tianpeng;Qin, Victor;Tang, Yujie;Li, Na
• 通讯作者: Li, Na

Distributed Information-Based Source Seeking

• DOI: 10.1109/tro.2023.3309099
• 发表时间: 2022-09
• 期刊: IEEE Transactions on Robotics
• 影响因子: 7.8
• 作者: Tianpeng Zhang;Victor Qin;Yujie Tang;Na Li

Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach

• DOI: 10.1109/infocom42981.2021.9488818
• 发表时间: 2021-02
• 期刊: IEEE INFOCOM 2021 - IEEE Conference on Computer Communications
• 作者: Junshan Zhang;Na Li;M. Dedeoglu