Collaborative Research: SHF: Medium: HERMES: On-Device Distributed Machine Learning via Model-Hardware Co-Design

协作研究：SHF：媒介：HERMES：通过模型硬件协同设计实现设备上分布式机器学习

• 批准号: 2107085
• 负责人: Diana Marculescu
• 金额: $ 56.4万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2107085&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; HERMES

Machine Learning (ML) is poised to become the most disruptive technology in modern society by changing all aspects of how humans interact with each other or with the world around them. To be effective, ML models must use vast amounts of data and must be built and updated efficiently wherever and whenever new data, devices, or users are available. To satisfy consumer needs or stringent device or environmental constraints, ML systems must respond fast and use minimum energy whenever possible, especially in the context of widely spread Internet-of-Things (IoT) devices. This project addresses this need by developing new approaches for distributed training that allows for fast and energy efficient training in the field, directly on IoT devices. The results of this project are poised to directly impact a wide array of applications, ranging from human mobility tracking and prediction, to real-time speech or language processing. Furthermore, the project aims to change how engineers are trained in a multidisciplinary fashion for dealing with the problem of efficiently designing distributed ML systems that respond in real-time and with low energy cost to availability of data, devices, or users. The project aims to develop a body of diverse research trainees, while expanding outreach to high-school and middle-school student populations. Given the unified interdisciplinary aspects of this work, its workforce development plan, and its industrial impact, this project enables wide collaboration among emerging or established engineers and industrial partners.Most training of ML models is done centrally in the cloud, thereby not satisfying user privacy concerns or response times, and becoming inapplicable if fast model updates are needed. While efficient on-device inference has been an intense focus of recent research, on-device distributed training and inference have not been addressed from response time and energy efficiency perspectives; this is particularly important for IoT, where the network plays a major part both in training and inference efficiency. To address these challenges, this project (dubbed HERMES) provides a unified multipronged approach for meeting real-time and energy constraints in an on-device distributed setting. HERMES ensures that ML methods and underlying hardware are co-designed, thereby addressing current challenges of private data sharing, communication overhead, or real-time and energy-efficient response of distributed ML. More specifically, Hermes includes: (i) a set of scalable approaches for hardware-aware real-time, energy efficient distributed training based on federated learning and distributed optimization that is robust to data and device variability; (ii) the co-design of ML model and hardware, comprising hyperparameter optimization that exploits hardware characteristics and identifies constraint-satisfying ML models, and hardware design exploration that efficiently finds constraint satisfying architectures; and (iii) an analysis and prototyping infrastructure for demonstrating the benefits of resulting ML systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习（ML）有望成为现代社会中最具颠覆性的技术，改变人类相互之间或与周围世界互动的各个方面。为了有效，机器学习模型必须使用大量数据，并且必须在任何时候、任何地方有新的数据、设备或用户可用时高效地构建和更新。为了满足消费者的需求或严格的设备或环境限制，机器学习系统必须尽可能快速响应并使用最少的能源，特别是在广泛分布的物联网（IoT）设备的背景下。 该项目通过开发分布式培训的新方法来满足这一需求，该方法允许直接在物联网设备上进行快速和节能的现场培训。该项目的结果将直接影响广泛的应用，从人类移动跟踪和预测到实时语音或语言处理。此外，该项目旨在改变工程师以多学科方式接受培训的方式，以处理有效设计分布式机器学习系统的问题，这些系统能够实时响应数据，设备或用户的可用性，并具有低能耗成本。该项目旨在培养一批多样化的研究受训人员，同时扩大对高中和初中学生的外联。鉴于这项工作的统一跨学科方面，其劳动力发展计划及其工业影响，该项目使新兴或成熟的工程师和工业合作伙伴之间能够进行广泛的合作。大多数ML模型的训练都在云中集中完成，因此无法满足用户隐私问题或响应时间，并且如果需要快速模型更新，则不适用。虽然高效的设备上推理一直是最近研究的重点，但从响应时间和能源效率的角度来看，设备上的分布式训练和推理还没有得到解决;这对物联网尤其重要，因为网络在训练和推理效率方面都起着重要作用。 为了应对这些挑战，该项目（称为爱马仕）提供了一个统一的多管齐下的方法，以满足实时和能源的限制，在设备上的分布式设置。爱马仕确保机器学习方法和底层硬件是共同设计的，从而解决了当前私有数据共享、通信开销或分布式机器学习的实时和节能响应的挑战。更具体地，爱马仕包括：（i）基于对数据和设备可变性鲁棒的联合学习和分布式优化的用于硬件感知实时、能量高效分布式训练的一组可扩展方法;（ii）ML模型和硬件的协同设计，包括利用硬件特性并识别满足约束的ML模型的超参数优化，和硬件设计探索，有效地找到满足约束的体系结构;以及（iii）该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的评估来支持。影响审查标准。

项目成果

MOHAWK: Mobility and Heterogeneity-Aware Dynamic Community Selection for Hierarchical Federated Learning

• DOI: 10.1145/3576842.3582378
• 发表时间: 2023-05
• 期刊: Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation
• 作者: Allen-Jasmin Farcas;Myungjin Lee;R. Kompella;Hugo Latapie;G. de Veciana;R. Marculescu

Demo Abstract: A Hardware Prototype Targeting Federated Learning with User Mobility and Device Heterogeneity

演示摘要：针对具有用户移动性和设备异构性的联邦学习的硬件原型

• DOI: 10.1145/3576842.3589160
• 发表时间: 2023
• 期刊: IoTDI
• 作者: Farcas, Allen-Jasmin;Marculescu, Radu
• 通讯作者: Marculescu, Radu

MobileTL: On-Device Transfer Learning with Inverted Residual Blocks

MobileTL：具有倒置残差块的设备上迁移学习

• DOI: 10.1609/aaai.v37i6.25874
• 发表时间: 2023
• 期刊: Proceedings of the AAAI Conference on Artificial Intelligence
• 作者: Chiang, Hung-Yueh;Frumkin, Natalia;Liang, Feng;Marculescu, Diana
• 通讯作者: Marculescu, Diana

Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP

• DOI: 10.1109/cvpr52729.2023.00682
• 发表时间: 2022-10
• 期刊: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Feng Liang;Bichen Wu;Xiaoliang Dai;Kunpeng Li;Yinan Zhao;Hang Zhang;Peizhao Zhang;Péter Vajda;D. Marculescu

SUGAR: Efficient Subgraph-Level Training via Resource-Aware Graph Partitioning

• DOI: 10.1109/tc.2023.3288755
• 发表时间: 2022-01
• 期刊: IEEE Transactions on Computers
• 影响因子: 3.7
• 作者: Zihui Xue;Yuedong Yang-;Mengtian Yang;R. Marculescu