Automated Customization of On-Device Inference for Quality-of-Experience Enhancement
Automated Customization of On-Device Inference for Quality-of-Experience Enhancement
复制标题
DOI：
10.1109/tc.2022.3208207
复制
发表时间：
2023-05-01
影响因子：
3.7
通讯作者：
Xu, Jie
中科院分区：
文献类型：
--
作者：
Bai, Yang;Chen, Lixing;Xu, Jie
智能应用的快速普及正在将深度学习(DL)功能推向移动设备。然而，设备容量、DNN性能和用户偏好的异构性使得为移动用户提供满意的体验质量(QOE)变得具有挑战性。本文研究了移动设备上DL推理的自动定制(称为设备上推理)，其目的是通过为不同使用场景的用户配置合适的DNN来提高用户的QOE。该方法的核心是DNN选择模块，它动态地学习用户的QOE模式，并利用学习到的知识识别最适合进行设备推理的DNN。它利用了一种在线学习算法NeuralUCB，该算法具有出色的泛化能力来处理各种用户QOE模式。我们还在NeuralUCB中嵌入了知识转移技术，以加快学习过程。然而，NeuralUCB经常向用户征求QOE评级，这带来了不可忽视的不便。为了解决这个问题，我们设计了反馈征集方案，在保持NeuralUCB的学习效率的同时，减少了QOE征集的数量。进一步研究了聚合QOE这一实用问题，以提高该框架的实用性。我们在合成数据和真实数据上都进行了实验。结果表明，该方法以较少的请求有效地学习了用户的QOE模式，并为移动设备提供了显著的QOE增强。
The rapid uptake of intelligent applications is pushing deep learning (DL) capabilities to mobile devices. However, the heterogeneities in device capacity, DNN performances, and user preferences make it challenging to provide satisfactory Quality of Experience (QoE) to mobile users. This paper studies automated customization for DL inference on mobile devices (termed as on-device inference), and our goal is to enhance user QoE by configuring the on-device inference with an appropriate DNN for users under different usage scenarios. The core of our method is a DNN selection module that learns user QoE patterns on-the-fly and identifies the best-fit DNN for on-device inference with the learned knowledge. It leverages an online learning algorithm, NeuralUCB, that has excellent generalization ability for handling various user QoE patterns. We also embed the knowledge transfer technique in NeuralUCB to expedite the learning process. However, NeuralUCB frequently solicits QoE ratings from users, which incurs non-negligible inconvenience. To address this problem, we design feedback solicitation schemes to reduce the number of QoE solicitations while maintaining the learning efficiency of NeuralUCB. A pragmatic problem, aggregated QoE, is further investigated to improve the practicality of our framework. We conduct experiments on both synthetic and real-world data. The results indicate that our method efficiently learns the user QoE pattern with few solicitations and provides drastic QoE enhancement for mobile devices.