Collaborative Research: SHF: Medium: Towards Harmonious Federated Intelligence in Heterogeneous Edge Computing via Data Migration

点击翻译按钮获取中文摘要

Edge computing has promoted a plethora of emerging applications that benefit people's daily life, such as smart cities, advanced manufacturing, and connected health. As the key enabler to this promising paradigm, the widely adopted Federated Learning (FL) algorithms can reshape the edge computing by offloading the training of large-scale data to nearby edges. To achieve federated intelligence with high accuracy and high efficiency, a major hindrance is data and system heterogeneity. When deploying FL algorithms to a practical edge computing system, the collected raw data may be corrupted and participating edges may experience different computational loads. Those heterogeneity issues significantly degrade the training efficiency and accuracy for achieving the ideal system performance. This project develops a Harmonious Federated Intelligence framework to allocate the collected data to its most favorable edge for training based on its intrinsic characteristics and required hardware resources. By enabling data migration across nearby heterogeneous edges, both learning models and heterogeneous data can be fed to the optimal edge for training without either wasting or overly exploiting hardware resources. The software-hardware co-design of harmonious federated intelligence fully unleashes the computational and communication potential of exiting edge computing infrastructures in transportation systems, manufacturing industries, home automation, and connected healthcare. This project seeks to broaden the scientific view of undergraduates and underrepresented students in the field of edge computing, machine learning, and data compression, and prepare them with the cross-disciplinary skills needed to succeed in the modern workforce.By introducing data-system-algorithm harmony, this project innovates the federated learning in heterogeneous edge computing to fundamentally tackle the data heterogeneity and unbalanced hardware resources usage. Given heterogeneous data samples with imbalanced feature spaces, Thrust 1 develops an imputation-based approach to complement missing features and values. Thrust 2 designs a Parallel Grow-and-Prune sparse training framework to schedule the sparse topology of learning models with joint consideration of both hardware resource budget and data characteristics. To enable efficient data migration, Thrust 3 develops adaptive data compression schemes, including both lossy and lossless compression algorithms, in different hardware settings. Thrust 4 proposes a fine-grained control mechanism for semi-asynchronous Vertical Federated Learning to adapt hardware resource reallocation and data migration, in order to minimize the impact of individual edge staleness due to system heterogeneity. The software-hardware co-design will be evaluated through data-driven simulation and experimental validation using an integrated platform consisting of a variety of edge devices featuring diverse computation and communication capabilities. To further validate the scalability, the team will develop a large-scale prototype on the NSF FABRIC testbed with core and edge nodes across the US.This project is jointly funded by the Software and Hardware Foundations (SHF) core research program in the Computing and Communication Foundations (CCF) Division and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.