SHF: Medium:DILSE: Codesigning Decentralized Incremental Learning System via Streaming Data Summarization on Edge

各种新兴领域，如互联自动驾驶汽车和智能家居分析，开启了物联网设备上的人工智能(AI)时代。然而，随着现代边缘设备的激增，其特征是有限的存储、异构的能力、动态的网络连接以及对数据隐私的日益关注，扩展和更新当前主流的集中式机器学习(ML)模型将变得不切实际，从而导致巨大的延迟、能量消耗和潜在的过时的性能下降的模型。因此，在设备上有效地处理本质上分散的数据流，即最接近它们的来源，而不在中央服务器上共享和积累原始训练样本，已经变得至关重要。虽然联邦学习(FL)已经出现来引入ML模型，但其全局模型在服务器端通过聚合局部模型来更新会导致模型收敛较差，并且需要在异质资源受限的边缘设备上的模型精度和可用资源之间进行折衷。此外，FL还没有被设计用于处理在边缘产生的流数据。该项目旨在通过结合集中式ML和分散式FL这两个世界的优点，为开发强大的ML模型开辟一种新的范式，从而推动释放人工智能改变人类生活的伟大承诺的前沿。这个项目的成果将导致新的课程材料跨越ML的几个领域(例如，分散的优化、计算机体系结构和边缘计算系统)和开放教育资源，旨在吸引不同的学生群体，并最终提供一个包容和创新的平台。该项目将考虑到数据的显著流传输和统计特征，以及广泛的设备和网络异构性，搭建集中式ML和分散式FL之间的桥梁。主要贡献是为新的去中心化ML训练设置奠定了严格的基础：(1)创建用于流数据的设备上摘要的新算法，以降低存储成本并改善处理延迟，同时保护隐私；(2)在通信网络中的设备的异构性下执行去中心化优化和通信；以及(3)共同设计节能的硬件架构和算法，用于加速具有实时推理的流数据汇总，并通过在真实异质边缘设备网络的边缘上进行流数据汇总来开发分散的增量学习，用于系统评估、验证和演示，同时促进绿色人工智能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Various emerging fields, such as connected autonomous vehicles and smart home analytics, have ushered in an era of Artificial Intelligence (AI) on Internet of Things devices. However, with the proliferation of modern edge devices characterized by limited storage, heterogeneous capabilities, dynamic network connection, and growing concerns of data privacy, it will become impractical to scale and update the current mainstream centralized machine learning (ML) models, leading to large latency delays, energy dissipation, and potentially outdated models with degraded performance. Hence, it has become paramount to efficiently process the inherently decentralized data streams on-device, i.e., closest to their sources without sharing and accumulating the raw training samples on a centralized server. While Federated learning (FL) has emerged to bring ML models, its global model updated at the server by aggregating local models can lead to poor model convergence and requires compromises between model accuracy and available resources on heterogeneous resource-constrained edge devices. Moreover, FL has not yet been designed for handling streaming data generated on the edge. This project aims to open up a new paradigm for developing powerful ML models by combining the best of both worlds of centralized ML and decentralized FL, and thus push forward the frontier of unleashing the great promise of AI to transform human life. The outcomes from this project will lead to new course materials spanning several areas of ML (e.g., decentralized optimization, computer architecture, and edge computing systems) and open-education resources that aim to attract diverse groups of students and eventually deliver a platform for inclusion and innovation.This project is to bridge Centralized ML and decentralized FL, considering the salient streaming and statistical characteristics of data combined with widespread device and network heterogeneity. The key contributions are to develop rigorous foundations for the new decentralized ML training setup in: (1) creating new algorithms for on-device summarization of streaming data to reduce memory cost and improve the processing latency, while preserving privacy; (2) performing decentralized optimization and communication under the heterogeneity of devices in a communication network; and (3) co-designing energy-efficient hardware architecture and algorithm for accelerating streaming data summarization with real-time inference, and developing decentralized incremental learning via streaming data summarization on the edge on a network of real heterogeneous edge devices for system evaluation, validation, and demonstration while promoting green AI.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.