SHF: Medium: Enabling Real-Time Federated Learning at the Edge: Algorithm and Circuit Co-Design
SHF: Medium: Enabling Real-Time Federated Learning at the Edge: Algorithm and Circuit Co-Design
批准号:
1955450
负责人:
Borivoje Nikolic
金额:
$100.0万
依托单位国家:
美国
项目类别:
Continuing Grant
财政年份:
2020
资助国家:
美国
项目状态:
已结题
起止时间:
2020-07-01 至 2024-06-30
中文摘要
机器学习应用程序正在渗透到许多新的领域,对隐私的日益关注正在将培训推向用户设备,而不是在云中的大型计算机上进行。在联邦学习范式中,模型训练是在使用用户私有本地数据集的大量分布式设备上执行的,而聚合模型是在云上组成的。这种方法在新算法的开发和分析以及优化硬件的协同设计方面提出了许多新的挑战。为了有效地部署联邦学习,边缘设备需要支持该计划感兴趣的许多其他方面,其中包括设备上的学习和使用私有数据的增量模型更新,这些通常是实时执行的。该项目建议建立一个端到端的实时联合学习框架,包括算法创新、软硬件协同设计和大规模技术中的高效硬件演示。同时,拟议的教育活动将促进工程师和科学家的发展,他们的专业知识涵盖从算法到数字系统实现的广泛领域。实时机器联邦学习的概念要求将理论算法方面与实际发展相结合。理论方面包括压缩模型大小、减少通信要求和性能评估。实用方面包括用于训练和推理的高效硬件,具有近内存计算的随机素描和硬件感知的神经网络设计。具体而言,它的目标是:(1)全面展示适合于各种场景部署的可扩展且节能的实时联邦学习架构;(2)通过测试芯片和基于云的FPGA仿真进行实验测量,以验证开发的系统模型;(3)以开源方式发布平台。这项工作的关键产品集成了研究和教育,包括可扩展的随机素描算法,用于设备上培训的节能和经济高效的机器学习加速器,以及用于硬件感知算法设计的快速准确的硬件建模基础设施。该奖项反映了美国国家科学基金会的法定使命,并通过使用基金会的知识价值和更广泛的影响审查标准进行评估,被认为值得支持。
英文摘要
Machine-learning applications are penetrating many new domains, and the increased concern for privacy is pushing training to the user devices, as opposed to being performed on large computers in the cloud. In the federated-learning paradigm, model training is performed on a large number of distributed devices with a user’s private local datasets, while the aggregate model is composed on the cloud. This approach poses a number of new challenges in both development and analysis of new algorithms and co-design of optimized hardware for efficient operation. For efficient deployment of federated learning, edge devices need to support many other aspects of interest for this program, among them on-device learning and incremental model updates with private data, often performed in real time. This project proposes to build an end-to-end, real-time federated-learning framework, ranging from algorithmic innovations, hardware-software co-design, and efficient hardware demonstrations in scaled technologies. Concurrently, the proposed education activities will enable the development of engineers and scientists whose expertise spans a broad range from algorithms to digital system implementation.The real-time machine federated-learning concept requires integration of theoretical algorithm aspects with their practical development. Theoretical aspects include the compression of model size, reduction in communication requirements and the assessment of performance. Practical aspects include efficient hardware for training and inference, randomized sketching with near-memory computation and hardware-aware neural network design. In particular, it aims to achieve: (1) A full demonstration of a scalable and energy-efficient real-time federated learning architecture suitable for deployment in various scenarios, (2) Experimental measurements via test chips and cloud-based FPGA simulation to validate the developed system models, (3) Release of the platform in the open source. The key products of this work integrate research and education and encompass scalable randomized sketching algorithms, energy- and cost-efficient machine-learning accelerators for on-device training, and fast-and-accurate hardware-modeling infrastructure for hardware-aware algorithm design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
期刊论文(6)
专著(0)
科研奖励(0)
会议论文
登录
查看更多内容
DOI:
10.48550/arxiv.2203.12786
发表时间:
2022-03
期刊:
ArXiv
影响因子:
--
作者:
[A. Zanette;M. Wainwright]
通讯作者:
A. Zanette;M. Wainwright
DOI:
10.48550/arxiv.2206.00796
发表时间:
2022-06
期刊:
影响因子:
--
作者:
[A. Zanette;M. Wainwright]
通讯作者:
A. Zanette;M. Wainwright
DOI:
--
发表时间:
2021-08
期刊:
影响因子:
--
作者:
[A. Zanette;M. Wainwright;E. Brunskill]
通讯作者:
A. Zanette;M. Wainwright;E. Brunskill
DOI:
10.1145/3579371.3589099
发表时间:
2023-06
期刊:
Proceedings of the 50th Annual International Symposium on Computer Architecture
影响因子:
--
作者:
[Dima Nikiforov;Shengjun Chris Dong;Chengyi Zhang;Seah Kim;B. Nikolić;Y. Shao]
通讯作者:
Dima Nikiforov;Shengjun Chris Dong;Chengyi Zhang;Seah Kim;B. Nikolić;Y. Shao
A new similarity measure for covariate shift with applications to nonparametric regression
协变量平移的新相似性度量及其在非参数回归中的应用
DOI:
--
发表时间:
2022
期刊:
Proceedings of the International Conference on Machine Learning
影响因子:
--
作者:
[Pathak, Reese, Ma, Cong, Wainwright, Martin J.]
通讯作者:
Wainwright, Martin J.
共 6 条
EARS: Energy- and Cost-Efficient Spectrum Utilization with Full-Duplex mm-wave Massive MIMO
-
批准号:1642920
-
项目类别:Standard Grant
-
资助金额:$130.0万
-
财政年份:2016
-
负责人:Borivoje Nikolic
-
依托单位:
EARS: Spectrum Sharing for Short-Latency Immersive Wireless Applications
-
批准号:1343398
-
项目类别:Standard Grant
-
资助金额:$70.0万
-
财政年份:2013
-
负责人:Borivoje Nikolic
-
依托单位:
Coding and System Design for Wireless Cooperative Relaying
-
批准号:1232318
-
项目类别:Continuing Grant
-
资助金额:$36.0万
-
财政年份:2012
-
负责人:Borivoje Nikolic
-
依托单位:
UC Berkeley Wireless Research Infrastructure Program
-
批准号:0403427
-
项目类别:Continuing Grant
-
资助金额:$80.13万
-
财政年份:2004
-
负责人:Borivoje Nikolic
-
依托单位:
CAREER: A Framework for Addressing Some Fundamental Challenges in Deeply Scaled CMOS Circuit Design
-
批准号:0238572
-
项目类别:Continuing Grant
-
资助金额:$40.0万
-
财政年份:2003
-
负责人:Borivoje Nikolic
-
依托单位:
海外基金