Collaborative Research: SHF: Medium: Towards Harmonious Federated Intelligence in Heterogeneous Edge Computing via Data Migration

边缘计算推动了智慧城市、先进制造、互联健康等一大批惠及人们日常生活的新兴应用。作为这一前景广阔的范例的关键使能者，被广泛采用的联邦学习(FL)算法可以通过将大规模数据的训练卸载到附近的边缘来重塑边缘计算。要实现高精度、高效率的联邦智能，一个主要的障碍是数据和系统的异构性。当将FL算法部署到实际的边缘计算系统时，收集的原始数据可能会被破坏，参与的边缘可能会经历不同的计算负载。这些异构性问题极大地降低了训练的效率和准确率，从而达到理想的系统性能。该项目开发了一个和谐的联邦智能框架，根据其内在特征和所需的硬件资源，将收集的数据分配到其最有利的训练优势。通过实现跨附近异质边缘的数据迁移，学习模型和异质数据都可以被馈送到最优边缘进行训练，而不会浪费或过度利用硬件资源。和谐联合智能的软硬件协同设计充分释放了交通系统、制造业、家庭自动化和互联医疗等领域现有边缘计算基础设施的计算和通信潜力。该项目旨在拓宽边缘计算、机器学习和数据压缩领域的本科生和未被充分代表的学生的科学视野，为他们在现代工作中取得成功所需的跨学科技能做好准备。通过引入数据-系统-算法协调，该项目创新了异质边缘计算中的联邦学习，从根本上解决了数据异构性和硬件资源使用不平衡的问题。对于具有不平衡特征空间的异质数据样本，推力1开发了一种基于归因的方法来补充缺失的特征和值。该算法综合考虑了硬件资源预算和数据特性，设计了一种并行增长剪枝稀疏训练框架来调度学习模型的稀疏拓扑。为了实现高效的数据迁移，推力3开发了不同硬件环境下的自适应数据压缩方案，包括有损和无损压缩算法。推力4提出了一种半异步垂直联合学习的细粒度控制机制，以适应硬件资源重新分配和数据迁移，以最大限度地减少由于系统异构性导致的个体边缘陈旧的影响。软硬件联合设计将通过数据驱动的模拟和实验验证进行评估，使用一个由各种具有不同计算和通信能力的边缘设备组成的集成平台。为了进一步验证可扩展性，该团队将在NSF交换矩阵试验台上开发一个大规模原型，在全美范围内具有核心和边缘节点。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Edge computing has promoted a plethora of emerging applications that benefit people's daily life, such as smart cities, advanced manufacturing, and connected health. As the key enabler to this promising paradigm, the widely adopted Federated Learning (FL) algorithms can reshape the edge computing by offloading the training of large-scale data to nearby edges. To achieve federated intelligence with high accuracy and high efficiency, a major hindrance is data and system heterogeneity. When deploying FL algorithms to a practical edge computing system, the collected raw data may be corrupted and participating edges may experience different computational loads. Those heterogeneity issues significantly degrade the training efficiency and accuracy for achieving the ideal system performance. This project develops a Harmonious Federated Intelligence framework to allocate the collected data to its most favorable edge for training based on its intrinsic characteristics and required hardware resources. By enabling data migration across nearby heterogeneous edges, both learning models and heterogeneous data can be fed to the optimal edge for training without either wasting or overly exploiting hardware resources. The software-hardware co-design of harmonious federated intelligence fully unleashes the computational and communication potential of exiting edge computing infrastructures in transportation systems, manufacturing industries, home automation, and connected healthcare. This project seeks to broaden the scientific view of undergraduates and underrepresented students in the field of edge computing, machine learning, and data compression, and prepare them with the cross-disciplinary skills needed to succeed in the modern workforce.By introducing data-system-algorithm harmony, this project innovates the federated learning in heterogeneous edge computing to fundamentally tackle the data heterogeneity and unbalanced hardware resources usage. Given heterogeneous data samples with imbalanced feature spaces, Thrust 1 develops an imputation-based approach to complement missing features and values. Thrust 2 designs a Parallel Grow-and-Prune sparse training framework to schedule the sparse topology of learning models with joint consideration of both hardware resource budget and data characteristics. To enable efficient data migration, Thrust 3 develops adaptive data compression schemes, including both lossy and lossless compression algorithms, in different hardware settings. Thrust 4 proposes a fine-grained control mechanism for semi-asynchronous Vertical Federated Learning to adapt hardware resource reallocation and data migration, in order to minimize the impact of individual edge staleness due to system heterogeneity. The software-hardware co-design will be evaluated through data-driven simulation and experimental validation using an integrated platform consisting of a variety of edge devices featuring diverse computation and communication capabilities. To further validate the scalability, the team will develop a large-scale prototype on the NSF FABRIC testbed with core and edge nodes across the US.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.