Deep Learning on Mobile Devices Through Neural Processing Units and Edge Computing
Deep Learning on Mobile Devices Through Neural Processing Units and Edge Computing
复制标题
DOI：
10.1109/infocom48880.2022.9796929
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Tianxiang Tan;G. Cao
中科院分区：
文献类型：
--
作者：
Tianxiang Tan;G. Cao
深度神经网络（DNN）已开始用于移动设备上的视频分析。为了减少运行DNN的延迟，许多移动设备配备了神经处理单元（NPU）。但是，由于NPU的资源限制，必须压缩这些DNN，以以准确性为代价提高处理速度。为了解决低准确性问题，我们为深度学习视频分析提出了一个基于信心的卸载（CBO）框架。主要的挑战是确定何时基于运行DNN的置信度返回NPU分类结果，以及何时将视频帧卸载到服务器中以进一步处理以提高准确性。我们首先确定使用现有置信度得分做出卸载决策的问题，并提出置信度得分校准技术以提高性能。然后，我们制定了CBO问题，在该问题中，目标是在某些时间限制下最大化准确性，并提出一种自适应解决方案，该解决方案确定哪种框架根据置信度评分和网络条件以哪些分辨率下载。通过真实的实施和广泛的评估，我们证明了所提出的解决方案可以显着超过其他方法。
Deep Neural Network (DNN) is becoming adopted for video analytics on mobile devices. To reduce the delay of running DNNs, many mobile devices are equipped with Neural Processing Units (NPU). However, due to the resource limitations of NPU, these DNNs have to be compressed to increase the processing speed at the cost of accuracy. To address the low accuracy problem, we propose a Confidence Based Offloading (CBO) framework for deep learning video analytics. The major challenge is to determine when to return the NPU classification result based on the confidence level of running the DNN, and when to offload the video frames to the server for further processing to increase the accuracy. We first identify the problem of using existing confidence scores to make offloading decisions, and propose confidence score calibration techniques to improve the performance. Then, we formulate the CBO problem where the goal is to maximize accuracy under some time constraint, and propose an adaptive solution that determines which frames to offload at what resolution based on the confidence score and the network condition. Through real implementations and extensive evaluations, we demonstrate that the proposed solution can significantly outperform other approaches.