权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

MLWiNS: Optimization and Coding Theory for Fast and Robust Wireless Distributed Learning

MLWiNS：快速、稳健的无线分布式学习的优化和编码理论

基本信息

批准号：
2003035
负责人：
Ramtin Pedarsani
金额：
$ 30万
依托单位：
University of California-Santa Barbara
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2003035&HistoricalAwards=false
关键词：
MLWiNS Optimization Coding Theory Fast

项目摘要

Wireless distributed learning systems can enable a variety of new applications including industrial automation, semantic learning, autonomous driving, health-care applications, etc. While wireless distributed learning brings about new opportunities, it faces two major challenges that severely limit its efficiency, reliability, and scalability: (1) Network heterogeneity, which is due to varying computational capabilities of edge devices. This challenge, also known as Straggler bottleneck, incurs large delays and failures due to computing nodes that are significantly slower than the rest; and (2) Communication Bottleneck, which is due to the massive amounts of raw or processed data that must be moved around the network. To tackle these bottlenecks, this project proposes techniques from coding theory and optimization theory to develop distributed learning algorithms with strong theoretical guarantees and empirical performance. Wireless distributed learning systems are driven by scaling out computations across many wireless edge nodes. There are, however, two major systems bottlenecks that arise: (1) Straggler Delay Bottleneck, which is due to the latency in waiting for slowest nodes to finish their tasks; (2) Data Shuffling Bottleneck, which is due to the massive amounts of data that must be moved among nodes. Moreover, there are privacy concerns about sharing sensitive local data, as well as vulnerabilities to adversarial attacks. This proposal aims to develop novel techniques from coding theory and optimization theory to tackle the mentioned bottlenecks and concerns. The project develops new "coded computing" algorithms for robust gradient aggregation, as well as new optimization algorithms for distributed learning. These algorithms are then used in two network settings to develop communication-efficient, straggler-resilient, and robust distributed learning frameworks: (i) a collaborative setting where a learning task is allocated to multiple edge nodes of the network. In this setting, data points can be encoded and offloaded to the edge nodes to provide resiliency against system bottlenecks; (ii) a federated setting where data points are gathered locally at edge devices and have to remain local due to privacy concerns.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

无线分布式学习系统可以实现各种新的应用，包括工业自动化、语义学习、自动驾驶、医疗保健应用等。虽然无线分布式学习带来了新的机遇，但它面临着两大挑战，严重限制了它的效率、可靠性和可扩展性：(1)网络异质性，这是由于边缘设备的计算能力不同造成的。这种挑战，也被称为离散瓶颈，由于计算节点比其他节点慢得多，会导致大量延迟和故障；(2)通信瓶颈，这是由于大量的原始或处理过的数据必须在网络上移动。为了解决这些瓶颈，本项目提出了编码理论和优化理论的技术，以开发具有强大理论保证和经验性能的分布式学习算法。无线分布式学习系统是由跨多个无线边缘节点的扩展计算驱动的。然而，出现了两个主要的系统瓶颈：(1)离散延迟瓶颈，这是由于等待最慢节点完成任务的延迟；(2)数据变换瓶颈，这是由于大量数据必须在节点之间移动。此外，共享敏感的本地数据还存在隐私问题，以及对抗性攻击的脆弱性。本提案旨在从编码理论和优化理论发展新技术来解决上述瓶颈和问题。该项目开发了用于鲁棒梯度聚合的新“编码计算”算法，以及用于分布式学习的新优化算法。然后在两种网络设置中使用这些算法来开发通信高效，离散弹性和健壮的分布式学习框架：(i)将学习任务分配给网络的多个边缘节点的协作设置。在这种情况下，数据点可以编码并卸载到边缘节点，以提供针对系统瓶颈的弹性；（ii）联邦设置，其中数据点在边缘设备上本地收集，并且由于隐私问题必须保持本地。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（11）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

CodedReduce: A Fast and Robust Framework for Gradient Aggregation in Distributed Learning

DOI：
10.1109/tnet.2021.3109097
发表时间：
2019-02
期刊：
IEEE/ACM Transactions on Networking
影响因子：
0
作者：
Amirhossein Reisizadeh;Saurav Prakash;Ramtin Pedarsani;A. Avestimehr
通讯作者：
Amirhossein Reisizadeh;Saurav Prakash;Ramtin Pedarsani;A. Avestimehr

Robust Federated Learning: The Case of Affine Distribution Shifts

DOI：
发表时间：
2020-06
期刊：
ArXiv
影响因子：
0
作者：
Amirhossein Reisizadeh;Farzan Farnia;Ramtin Pedarsani;A. Jadbabaie
通讯作者：
Amirhossein Reisizadeh;Farzan Farnia;Ramtin Pedarsani;A. Jadbabaie

Provably Private Distributed Averaging Consensus: An Information-Theoretic Approach

DOI：
10.1109/tit.2023.3300711
发表时间：
2022-02
期刊：
IEEE Transactions on Information Theory
影响因子：
2.5
作者：
Mohammad Fereydounian;Aryan Mokhtari;Ramtin Pedarsani;Hamed Hassani
通讯作者：
Mohammad Fereydounian;Aryan Mokhtari;Ramtin Pedarsani;Hamed Hassani

Asymptotic Behavior of Adversarial Training in Binary Linear Classification

二元线性分类中对抗训练的渐近行为

DOI：
10.1109/isit50566.2022.9834717
发表时间：
2022
期刊：
2022 IEEE International Symposium on Information Theory (ISIT
影响因子：
0
作者：
Taheri, Hossein;Pedarsani, Ramtin;Thrampoulidis, Christos
通讯作者：
Thrampoulidis, Christos

Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity

DOI：
10.1109/jsait.2022.3205475
发表时间：
2020-12
期刊：
IEEE Journal on Selected Areas in Information Theory
影响因子：
0
作者：
Amirhossein Reisizadeh;Isidoros Tziotis;Hamed Hassani;Aryan Mokhtari;Ramtin Pedarsani
通讯作者：
Amirhossein Reisizadeh;Isidoros Tziotis;Hamed Hassani;Aryan Mokhtari;Ramtin Pedarsani