Collaborative Research: SHF: Medium: Towards Harmonious Federated Intelligence in Heterogeneous Edge Computing via Data Migration

边缘计算促进了大量有益于人们日常生活的新兴应用，如智慧城市、先进制造和互联健康。作为这一有前途的范例的关键推动者，广泛采用的联邦学习（FL）算法可以通过将大规模数据的训练卸载到附近的边缘来重塑边缘计算。要实现高精度和高效率的联邦智能，一个主要的障碍是数据和系统的异构性。当将FL算法部署到实际的边缘计算系统时，收集的原始数据可能会被破坏，并且参与的边缘可能会经历不同的计算负载。这些异构性问题显著降低了实现理想系统性能的训练效率和准确性。本项目开发了一个和谐联邦智能框架，根据其内在特性和所需的硬件资源，将收集到的数据分配到其最有利的边缘进行训练。通过实现跨附近异构边缘的数据迁移，学习模型和异构数据都可以被馈送到最佳边缘进行训练，而不会浪费或过度利用硬件资源。和谐联邦智能的软硬件协同设计充分释放了交通系统、制造业、家庭自动化和互联医疗保健中现有边缘计算基础设施的计算和通信潜力。该项目旨在拓宽本科生和边缘计算、机器学习和数据压缩领域代表性不足的学生的科学视野，并使他们具备在现代劳动力中取得成功所需的跨学科技能。通过引入数据-系统-算法和谐，该项目创新了异构边缘计算中的联邦学习，从根本上解决数据异构和硬件资源使用不平衡的问题。给定具有不平衡特征空间的异构数据样本，Thrust 1开发了一种基于估算的方法来补充缺失的特征和值。Thrust 2设计了并行生长和修剪稀疏训练框架，以调度学习模型的稀疏拓扑，同时考虑硬件资源预算和数据特性。为了实现高效的数据迁移，Thrust 3开发了自适应数据压缩方案，包括在不同硬件设置下的有损和无损压缩算法。Thrust 4提出了一种用于半异步垂直联邦学习的细粒度控制机制，以适应硬件资源重新分配和数据迁移，以最大限度地减少由于系统异构性而导致的个体边缘陈旧的影响。软硬件协同设计将通过数据驱动的仿真和实验验证进行评估，使用由各种边缘设备组成的集成平台，具有不同的计算和通信能力。为了进一步验证可扩展性，该团队将在NSF FABRIC测试平台上开发一个大规模原型，其核心和边缘节点遍布美国。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Edge computing has promoted a plethora of emerging applications that benefit people's daily life, such as smart cities, advanced manufacturing, and connected health. As the key enabler to this promising paradigm, the widely adopted Federated Learning (FL) algorithms can reshape the edge computing by offloading the training of large-scale data to nearby edges. To achieve federated intelligence with high accuracy and high efficiency, a major hindrance is data and system heterogeneity. When deploying FL algorithms to a practical edge computing system, the collected raw data may be corrupted and participating edges may experience different computational loads. Those heterogeneity issues significantly degrade the training efficiency and accuracy for achieving the ideal system performance. This project develops a Harmonious Federated Intelligence framework to allocate the collected data to its most favorable edge for training based on its intrinsic characteristics and required hardware resources. By enabling data migration across nearby heterogeneous edges, both learning models and heterogeneous data can be fed to the optimal edge for training without either wasting or overly exploiting hardware resources. The software-hardware co-design of harmonious federated intelligence fully unleashes the computational and communication potential of exiting edge computing infrastructures in transportation systems, manufacturing industries, home automation, and connected healthcare. This project seeks to broaden the scientific view of undergraduates and underrepresented students in the field of edge computing, machine learning, and data compression, and prepare them with the cross-disciplinary skills needed to succeed in the modern workforce.By introducing data-system-algorithm harmony, this project innovates the federated learning in heterogeneous edge computing to fundamentally tackle the data heterogeneity and unbalanced hardware resources usage. Given heterogeneous data samples with imbalanced feature spaces, Thrust 1 develops an imputation-based approach to complement missing features and values. Thrust 2 designs a Parallel Grow-and-Prune sparse training framework to schedule the sparse topology of learning models with joint consideration of both hardware resource budget and data characteristics. To enable efficient data migration, Thrust 3 develops adaptive data compression schemes, including both lossy and lossless compression algorithms, in different hardware settings. Thrust 4 proposes a fine-grained control mechanism for semi-asynchronous Vertical Federated Learning to adapt hardware resource reallocation and data migration, in order to minimize the impact of individual edge staleness due to system heterogeneity. The software-hardware co-design will be evaluated through data-driven simulation and experimental validation using an integrated platform consisting of a variety of edge devices featuring diverse computation and communication capabilities. To further validate the scalability, the team will develop a large-scale prototype on the NSF FABRIC testbed with core and edge nodes across the US.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.