Collaborative Research: SHF: Medium: Towards Harmonious Federated Intelligence in Heterogeneous Edge Computing via Data Migration

边缘计算推动了大量新兴应用的发展，造福于人们的日常生活，如智慧城市、先进制造和互联健康。作为这一有前途的范例的关键推动者，广泛采用的联邦学习（FL）算法可以通过将大规模数据的训练转移到附近的边缘来重塑边缘计算。要实现高精度、高效率的联邦智能，数据和系统的异构性是一个主要的障碍。当将FL算法部署到实际边缘计算系统时，收集的原始数据可能会损坏，并且参与边缘可能会经历不同的计算负载。这些异构性问题显著降低了训练效率和准确性，无法达到理想的系统性能。该项目开发了一个和谐的联邦智能框架，根据数据的内在特征和所需的硬件资源，将收集到的数据分配到最有利的训练优势。通过支持跨附近异构边缘的数据迁移，学习模型和异构数据都可以馈送到最优边缘进行训练，而不会浪费或过度利用硬件资源。和谐联邦智能的软硬件协同设计充分释放了交通系统、制造业、家庭自动化和互联医疗中现有边缘计算基础设施的计算和通信潜力。该项目旨在拓宽本科生和未被充分代表的学生在边缘计算、机器学习和数据压缩领域的科学视野，并为他们提供在现代劳动力中取得成功所需的跨学科技能。本项目通过引入数据-系统-算法和谐，创新异构边缘计算中的联邦学习，从根本上解决数据异构和硬件资源使用不平衡的问题。针对具有不平衡特征空间的异构数据样本，Thrust 1开发了一种基于假设的方法来补充缺失的特征和值。Thrust 2设计了一个并行的Grow-and-Prune稀疏训练框架来调度学习模型的稀疏拓扑，同时考虑硬件资源预算和数据特征。为了实现高效的数据迁移，Thrust 3开发了自适应数据压缩方案，包括在不同硬件设置下的有损和无损压缩算法。Thrust 4提出了一种用于半异步垂直联邦学习的细粒度控制机制，以适应硬件资源重新分配和数据迁移，从而最大限度地减少由于系统异构而导致的单个边缘过时的影响。软件-硬件协同设计将通过数据驱动的模拟和实验验证进行评估，使用由各种具有不同计算和通信能力的边缘设备组成的集成平台。为了进一步验证可扩展性，该团队将在美国的NSF FABRIC测试平台上开发一个大型原型，其中包含核心和边缘节点。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Edge computing has promoted a plethora of emerging applications that benefit people's daily life, such as smart cities, advanced manufacturing, and connected health. As the key enabler to this promising paradigm, the widely adopted Federated Learning (FL) algorithms can reshape the edge computing by offloading the training of large-scale data to nearby edges. To achieve federated intelligence with high accuracy and high efficiency, a major hindrance is data and system heterogeneity. When deploying FL algorithms to a practical edge computing system, the collected raw data may be corrupted and participating edges may experience different computational loads. Those heterogeneity issues significantly degrade the training efficiency and accuracy for achieving the ideal system performance. This project develops a Harmonious Federated Intelligence framework to allocate the collected data to its most favorable edge for training based on its intrinsic characteristics and required hardware resources. By enabling data migration across nearby heterogeneous edges, both learning models and heterogeneous data can be fed to the optimal edge for training without either wasting or overly exploiting hardware resources. The software-hardware co-design of harmonious federated intelligence fully unleashes the computational and communication potential of exiting edge computing infrastructures in transportation systems, manufacturing industries, home automation, and connected healthcare. This project seeks to broaden the scientific view of undergraduates and underrepresented students in the field of edge computing, machine learning, and data compression, and prepare them with the cross-disciplinary skills needed to succeed in the modern workforce.By introducing data-system-algorithm harmony, this project innovates the federated learning in heterogeneous edge computing to fundamentally tackle the data heterogeneity and unbalanced hardware resources usage. Given heterogeneous data samples with imbalanced feature spaces, Thrust 1 develops an imputation-based approach to complement missing features and values. Thrust 2 designs a Parallel Grow-and-Prune sparse training framework to schedule the sparse topology of learning models with joint consideration of both hardware resource budget and data characteristics. To enable efficient data migration, Thrust 3 develops adaptive data compression schemes, including both lossy and lossless compression algorithms, in different hardware settings. Thrust 4 proposes a fine-grained control mechanism for semi-asynchronous Vertical Federated Learning to adapt hardware resource reallocation and data migration, in order to minimize the impact of individual edge staleness due to system heterogeneity. The software-hardware co-design will be evaluated through data-driven simulation and experimental validation using an integrated platform consisting of a variety of edge devices featuring diverse computation and communication capabilities. To further validate the scalability, the team will develop a large-scale prototype on the NSF FABRIC testbed with core and edge nodes across the US.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.