Efficient Distributed Inference of Deep Neural Networks via Restructuring and Pruning
Efficient Distributed Inference of Deep Neural Networks via Restructuring and Pruning
复制标题
DOI：
10.1609/aaai.v37i6.25815
复制
发表时间：
2023-06
影响因子：
--
通讯作者：
A. Abdi;Saeed Rashidi;F. Fekri;T. Krishna
中科院分区：
文献类型：
--
作者：
A. Abdi;Saeed Rashidi;F. Fekri;T. Krishna
在本文中，我们考虑在多个处理节点（也称为工作节点）上并行实现已训练的深度模型。具体来说，我们研究如何将深度模型划分为多个并行子模型，每个子模型都由工作人员有效执行。由于工作线程之间的同步和数据传输造成的延迟会对并行实现的性能产生负面影响，因此希望并行子模型之间的相互依赖性最小。为了实现这一目标，我们建议重新排列神经网络中的神经元，对它们进行分区（不改变神经网络的一般拓扑），并修改权重，以便在工作人员的计算和通信约束下最小化子模型之间的相互依赖性，同时最小化其对模型性能的影响。我们提出了 RePurpose，一种分层模型重组和修剪技术，可以保证整体并行化模型的性能。为了有效地应用 RePurpose，我们提出了一种基于 L0 优化和 Munkres 分配算法的方法。我们表明，与现有方法相比，RePurpose 通过并行实现在通信和计算复杂度方面显着提高了分布式推理的效率。
In this paper, we consider the parallel implementation of an already-trained deep model on multiple processing nodes (a.k.a. workers). Specifically, we investigate as to how a deep model should be divided into several parallel sub-models, each of which is executed efficiently by a worker. Since latency due to synchronization and data transfer among workers negatively impacts the performance of the parallel implementation, it is desirable to have minimum interdependency among parallel sub-models. To achieve this goal, we propose to rearrange the neurons in the neural network, partition them (without changing the general topology of the neural network), and modify the weights such that the interdependency among sub-models is minimized under the computations and communications constraints of the workers while minimizing its impact on the performance of the model. We propose RePurpose, a layer-wise model restructuring and pruning technique that guarantees the performance of the overall parallelized model. To efficiently apply RePurpose, we propose an approach based on L0 optimization and the Munkres assignment algorithm. We show that, compared to the existing methods, RePurpose significantly improves the efficiency of the distributed inference via parallel implementation, both in terms of communication and computational complexity.