FedDebug: Systematic Debugging for Federated Learning Applications
FedDebug: Systematic Debugging for Federated Learning Applications
复制标题
DOI：
10.1109/icse48619.2023.00053
复制
发表时间：
2023-01
期刊：
影响因子：
--
通讯作者：
Waris Gill;A. Anwar;Muhammad Ali Gulzar
中科院分区：
文献类型：
--
作者：
Waris Gill;A. Anwar;Muhammad Ali Gulzar
在联邦学习（FL）中，客户端独立训练本地模型，并与中央聚合器共享以构建全局模型。不允许访问客户的数据和协作培训使FL对具有数据隐私问题的应用程序具有吸引力，例如医学成像。然而，这些FL特性对调试提出了前所未有的挑战。当一个全球模型的表现恶化时，识别负责任的轮次和客户是一个主要的痛点。开发人员求助于对客户端子集进行试错调试，希望提高全局模型的准确性，或者让未来的FL轮重新调整模型，这既耗时又昂贵。我们设计了一个系统的故障定位框架，Fedde-bug，在两个新的方面推进FL调试。首先，Feddebug通过利用记录和重放技术来构建反映现场FL的仿真，从而实现FL中实时协作训练的交互式调试。Feddebug的断点可以帮助检查FL状态（轮、客户端和全局模型），并在轮和客户端模型之间无缝移动，从而实现细粒度的分步检查。其次，Feddebug自动识别负责降低全局模型性能的客户端，而无需任何测试数据和标签-这两者对于现有的调试技术都是必不可少的。Feddebug的优势在于将差分测试与神经元激活相结合，以确定偏离正常行为的客户端。Feddebug在查找单个错误客户端时达到100%的准确率，在查找多个错误客户端时达到90.3%的准确率。Feddebug的交互式调试在训练过程中会产生1.2%的开销，而它只需要一轮训练时间的2.1%就可以定位错误的客户端。通过FederWorks，我们为联邦学习带来了有效的调试实践，提高了FL应用程序开发人员的质量和生产力。
In Federated Learning (FL), clients independently train local models and share them with a central aggregator to build a global model. Impermissibility to access clients' data and collaborative training make FL appealing for applications with data-privacy concerns, such as medical imaging. However, these FL characteristics pose unprecedented challenges for debugging. When a global model's performance deteriorates, identifying the responsible rounds and clients is a major pain point. Developers resort to trial-and-error debugging with subsets of clients, hoping to increase the global model's accuracy or let future FL rounds retune the model, which are time-consuming and costly. We design a systematic fault localization framework, Fedde-bug,that advances the FL debugging on two novel fronts. First, Feddebug enables interactive debugging of realtime collaborative training in FL by leveraging record and replay techniques to construct a simulation that mirrors live FL. Feddebug'sbreakpoint can help inspect an FL state (round, client, and global model) and move between rounds and clients' models seam-lessly, enabling a fine-grained step-by-step inspection. Second, Feddebug automatically identifies the client(s) responsible for lowering the global model's performance without any testing data and labels-both are essential for existing debugging techniques. Feddebug's strengths come from adapting differential testing in conjunction with neuron activations to determine the client(s) deviating from normal behavior. Feddebug achieves 100% accuracy in finding a single faulty client and 90.3% accuracy in finding multiple faulty clients. Feddebug's interactive de-bugging incurs 1.2% overhead during training, while it localizes a faulty client in only 2.1% of a round's training time. With FedDebug,we bring effective debugging practices to federated learning, improving the quality and productivity of FL application developers.