FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
FedScale: Benchmarking Model and System Performance of Federated Learning at Scale
复制标题
DOI：
--
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Fan Lai;Yinwei Dai;Sanjay Sri Vallabh Singapuram;Jiachen Liu;Xiangfeng Zhu;H. Madhyastha;Mosharaf Chowdhury
中科院分区：
文献类型：
--
作者：
Fan Lai;Yinwei Dai;Sanjay Sri Vallabh Singapuram;Jiachen Liu;Xiangfeng Zhu;H. Madhyastha;Mosharaf Chowdhury
我们提出了联邦学习（FL）基准测试套件，具有现实数据集和可扩展的运行时，以实现可重复的FL研究。FedScale数据集涵盖了广泛的关键FL任务，从图像分类和目标检测到语言建模和语音识别。每个数据集都有一个统一的评估协议，使用真实世界的数据分割和评估指标。为了重现真实的FL行为，FedScale包含了一个可伸缩和可扩展的运行时。它提供了高级api来实现FL算法，在不同的硬件和软件后端大规模部署它们，并大规模评估它们，所有这些都只需要最少的开发人员的努力。我们将两者结合起来进行系统的基准测试实验，并强调在FL中进行异构感知协同优化的潜在机会。FedScale是开源的，并由来自不同机构的贡献者在http://fedscale.ai上积极维护。我们欢迎来自社区的反馈和贡献。
We present FedScale, a federated learning (FL) benchmarking suite with realistic datasets and a scalable runtime to enable reproducible FL research. FedScale datasets encompass a wide range of critical FL tasks, ranging from image classification and object detection to language modeling and speech recognition. Each dataset comes with a unified evaluation protocol using real-world data splits and evaluation metrics. To reproduce realistic FL behavior, FedScale contains a scalable and extensible runtime. It provides high-level APIs to implement FL algorithms, deploy them at scale across diverse hardware and software backends, and evaluate them at scale, all with minimal developer efforts. We combine the two to perform systematic benchmarking experiments and highlight potential opportunities for heterogeneity-aware co-optimizations in FL. FedScale is open-source and actively maintained by contributors from different institutions at http://fedscale.ai. We welcome feedback and contributions from the community.