A Bargaining Game for Personalized, Energy Efficient Split Learning over Wireless Networks
A Bargaining Game for Personalized, Energy Efficient Split Learning over Wireless Networks
复制标题
DOI：
10.1109/wcnc55385.2023.10118601
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Minsu Kim;Alexander C. DeRieux;W. Saad
中科院分区：
文献类型：
--
作者：
Minsu Kim;Alexander C. DeRieux;W. Saad
分裂学习（SL）是一种新兴的分布式学习框架，可以减轻联邦学习的计算和无线通信开销。它将机器学习模型分为设备端模型和服务器端模型。设备只训练它们分配的模型，并将剪切层的激活传输到服务器。然而，SL可能导致数据泄漏，因为服务器可以使用输入和中间激活之间的相关性来重建输入数据。虽然为设备端模型分配更多的层可以降低数据泄漏的可能性，但这将导致资源受限的设备消耗更多的能量，并为服务器提供更多的训练时间。此外，跨设备的非iid数据集将降低收敛速度，导致训练时间增加。本文提出了一种新的个性化SL框架。对于这个框架，开发了一种新的方法来选择切割层，可以优化计算和无线传输的能量消耗，训练时间和数据隐私之间的权衡。在所考虑的框架中，每个设备个性化其设备端模型，以减轻非iid数据集，同时共享相同的服务器端模型进行泛化。为了平衡计算和无线传输的能量消耗，训练时间和数据隐私，制定了一个多人讨价还价问题，以找到设备和服务器之间的最佳切割层。为了解决该问题，利用二分法得到了Kalai-Smorodinsky讨价还价解（KSBS），并进行了可行性检验。仿真结果表明，基于KSBS分割层的个性化SL框架能够在能量消耗、训练时间和数据隐私之间取得最优的和效用，并且对非iid数据集具有鲁棒性.
Split learning (SL) is an emergent distributed learning framework which can mitigate the computation and wireless communication overhead of federated learning. It splits a machine learning model into a device-side model and a server-side model at a cut layer. Devices only train their allocated model and transmit the activations of the cut layer to the server. However, SL can lead to data leakage as the server can reconstruct the input data using the correlation between the input and intermediate activations. Although allocating more layers to a device-side model can reduce the possibility of data leakage, this will lead to more energy consumption for resource-constrained devices and more training time for the server. Moreover, non-iid datasets across devices will reduce the convergence rate leading to increased training time. In this paper, a new personalized SL framework is proposed. For this framework, a novel approach for choosing the cut layer that can optimize the tradeoff between the energy consumption for computation and wireless transmission, training time, and data privacy is developed. In the considered framework, each device personalizes its device-side model to mitigate non-iid datasets while sharing the same server-side model for generalization. To balance the energy consumption for computation and wireless transmission, training time, and data privacy, a multiplayer bargaining problem is formulated to find the optimal cut layer between devices and the server. To solve the problem, the Kalai-Smorodinsky bargaining solution (KSBS) is obtained using the bisection method with the feasibility test. Simulation results show that the proposed personalized SL framework with the cut layer from the KSBS can achieve the optimal sum utilities by balancing the energy consumption, training time, and data privacy, and it is also robust to non-iid datasets.