All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management
All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management
复制标题
DOI：
10.1145/3508352.3549379
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Yifan Gong;Zheng Zhan;Pu Zhao;Yushu Wu;Chaoan Wu;Caiwen Ding;Weiwen Jiang;Minghai Qin;Yanzhi Wang
中科院分区：
文献类型：
--
作者：
Yifan Gong;Zheng Zhan;Pu Zhao;Yushu Wu;Chaoan Wu;Caiwen Ding;Weiwen Jiang;Minghai Qin;Yanzhi Wang
在边缘设备上部署深度神经网络(DNN)的过程中，许多研究工作都致力于有限的硬件资源。然而，动态功率管理的影响却鲜有人关注。由于边缘设备通常只有电池的能量预算(而不是服务器或工作站上的几乎无限的能量支持)，它们的动态电源管理经常改变执行频率，如在广泛使用的动态电压和频率调整(DVFS)技术中一样。这导致推理速度性能高度不稳定，特别是对于计算密集型的DNN模型，这可能会损害用户体验，浪费硬件资源。我们首先发现了这个问题，然后提出了All-in-One，一个非常有代表性的剪枝框架，用于使用DVFS进行动态电源管理。该框架仅使用一组模型权重和软掩码(连同其他可忽略存储的辅助参数)来表示不同剪枝比的多个模型。通过将模型重新配置为特定执行频率(和电压)下的相应剪枝比，我们能够获得稳定的推理速度，即在不同执行频率下的速度性能差异尽可能小。实验表明，该方法不仅对不同剪枝比的多个模型具有较高的准确率，而且降低了它们在不同频率下的推理延迟方差，只需要一个模型和一个软掩码就可以达到最小的内存消耗。
During the deployment of deep neural networks (DNNs) on edge devices, many research efforts are devoted to the limited hardware resource. However, little attention is paid to the influence of dynamic power management. As edge devices typically only have a budget of energy with batteries (rather than almost unlimited energy support on servers or workstations), their dynamic power management often changes the execution frequency as in the widely-used dynamic voltage and frequency scaling (DVFS) technique. This leads to highly unstable inference speed performance, especially for computation-intensive DNN models, which can harm user experience and waste hardware resources. We firstly identify this problem and then propose All-in-One, a highly representative pruning framework to work with dynamic power management using DVFS. The framework can use only one set of model weights and soft masks (together with other auxiliary parameters of negligible storage) to represent multiple models of various pruning ratios. By re-configuring the model to the corresponding pruning ratio for a specific execution frequency (and voltage), we are able to achieve stable inference speed, i.e., keeping the difference in speed performance under various execution frequencies as small as possible. Our experiments demonstrate that our method not only achieves high accuracy for multiple models of different pruning ratios, but also reduces their variance of inference latency for various frequencies, with minimal memory consumption of only one model and one soft mask.