Collaborative Research: CNS Core: Small: Towards Automated and QoE-driven Machine Learning Model Selection for Edge Inference

边缘设备，如手机、无人机和机器人，已经成为深度神经网络（DNN）推理越来越重要的平台。对于边缘设备，从许多可能性中选择最佳DNN模型对于最大化用户体验质量（QoE）至关重要，但这受到边缘设备高度异质性和不断变化的使用场景的显著挑战。目前的实践通常为许多或所有边缘设备选择单个DNN模型，这最多只能为一小部分用户提供满意的QoE。另外，特定于设备的DNN模型优化是耗时的，并且不能扩展到各种边缘设备。此外，现有方法侧重于优化边缘推断的某个客观度量，这可能无法转化为对用户实际QoE的改进。通过利用机器学习的预测能力并保持用户处于循环中，该项目提出了一个用于qoe最优边缘推理的自动化和可扩展的设备级DNN模型选择引擎。具体来说，该项目包括两个重点：首先，它利用在线学习来预测每个边缘设备的QoE，自动化部署阶段DNN模型选择；其次，构建运行时QoE预测器，在给定运行时上下文信息的情况下自动选择最优DNN模型。该项目代表了对DNN模型优化当前实践的重要背离和必要补充。它可以将支持深度神经网络的智能的好处带给具有最佳QoE的更多资源受限的边缘设备。此外，它还为边缘推理提供了新的观察、见解和原则，促进了深度神经网络模型设计向以用户为中心的新范式的转变。该项目还为改进课程设计和吸引学生，特别是代表性不足的少数民族学生参与科学、技术、工程和数学领域提供了新的机会。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Edge devices, such as mobile phones, drones and robots, have been emerging as an increasingly more important platform for deep neural network (DNN) inference. For an edge device, selecting an optimal DNN model out of many possibilities is crucial for maximizing the user’s quality of experience (QoE), but this is significantly challenged by the high degree of heterogeneity in edge devices and constant-changing usage scenarios. The current practice commonly selects a single DNN model for many or all edge devices, which can only provide a satisfactory QoE for a small fraction of users at best. Alternatively, device-specific DNN model optimization is time-consuming and not scalable to a large diversity of edge devices. Moreover, the existing approaches focus on optimizing a certain objective metric for edge inference, which may not translate into improvement of the actual QoE for users. By leveraging the predictive power of machine learning and keeping users in a loop, this project proposes an automated and scalable device-level DNN model selection engine for QoE-optimal edge inference. Specifically, this project includes two thrusts: first, it exploits online learning to predict QoE for each edge device, automating deployment-stage DNN model selection; and second, it builds a runtime QoE predictor and automatically selects an optimal DNN model given runtime contextual information.This project represents an important departure from and an essential complement to the current practices in DNN model optimization. It can bring the benefits of DNN-enabled intelligence to many more resource-constrained edge devices with an optimal QoE. Additionally, it provides novel observations, insights and principles for edge inference, catalyzing the transformation of the design of DNN models into a new user-centric paradigm. This project also enables new opportunities to improve curriculum design and attract students, especially under-represented minorities, to engage in science, technology, engineering, and mathematics fields.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.