Improving QoE of Deep Neural Network Inference on Edge Devices: A Bandit Approach
Improving QoE of Deep Neural Network Inference on Edge Devices: A Bandit Approach
复制标题
DOI：
10.1109/jiot.2022.3182728
复制
发表时间：
2022-11
影响因子：
10.6
通讯作者：
Bingqian Lu;Jianyi Yang;Jie Xu;Shaolei Ren
中科院分区：
文献类型：
--
作者：
Bingqian Lu;Jianyi Yang;Jie Xu;Shaolei Ren
边缘设备，特别是移动设备，已成为深度神经网络 (DNN) 推理日益重要的平台。通常，使用不同架构和/或压缩方案生成的多个轻量级 DNN 模型可以适合一台设备，因此选择最佳模型对于最大化用户的边缘推理体验质量 (QoE) 至关重要。鉴于边缘设备极其多样化，现有的设备感知 DNN 优化方法通常非常耗时且不可扩展。更重要的是，他们专注于优化标准性能指标（例如准确性和延迟），这可能不会转化为用户实际主观 QoE 的改善。在本文中，我们提出了一种新颖的自动化且以用户为中心的 DNN 选择引擎，称为 $\mathsf {Aquaman}$ ，它将用户保持在闭环中，并利用他们的 QoE 反馈来指导 DNN 选择决策。 $\mathsf {Aquaman}$的核心是基于神经网络的QoE预测器，并且持续在线更新。此外，我们使用神经老虎机学习来平衡利用和探索，并具有可证明有效的 QoE 性能。最后，我们在 15 个用户的实验研究和综合模拟中评估了 $\mathsf {Aquaman}$ ，证明了 $\mathsf {Aquaman}$ 的有效性。
Edge devices, including, in particular, mobile devices, have been emerging as an increasingly more important platform for deep neural network (DNN) inference. Typically, multiple lightweight DNN models generated using different architectures and/or compression schemes can fit into a device, thus selecting an optimal one is crucial in order to maximize the users’ Quality of Experience (QoE) for edge inference. The existing approaches to device-aware DNN optimization are usually time consuming and not scalable in view of extremely diverse edge devices. More importantly, they focus on optimizing standard performance metrics (e.g., accuracy and latency), which may not translate into improvement of the users’ actual subjective QoE. In this article, we propose a novel automated and user-centric DNN selection engine, called $\mathsf {Aquaman}$ , which keeps users into a closed loop and leverages their QoE feedback to guide DNN selection decisions. The core of $\mathsf {Aquaman}$ is a neural network-based QoE predictor, which is continuously updated online. Additionally, we use neural bandit learning to balance exploitation and exploration, with a provably efficient QoE performance. Finally, we evaluate $\mathsf {Aquaman}$ on a 15-user experimental study as well as synthetic simulations, demonstrating the effectiveness of $\mathsf {Aquaman}$ .