Decentralized Application-Level Adaptive Scheduling for Multi-Instance DNNs on Open Mobile Devices
Decentralized Application-Level Adaptive Scheduling for Multi-Instance DNNs on Open Mobile Devices
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Hsin-Hsuan Sung;Jou-An Chen;Weiguo Niu;Jiexiong Guan;Bin Ren;Xipeng Shen
中科院分区：
文献类型：
--
作者：
Hsin-Hsuan Sung;Jou-An Chen;Weiguo Niu;Jiexiong Guan;Bin Ren;Xipeng Shen
随着越来越多的应用程序采用人工智能，多个深度神经网络（DNN）驱动的应用程序可以在移动终端上同时运行的情况越来越普遍。本文探讨了这种多实例DNN场景中的调度，在一般的开放式移动的系统（例如，常见的智能手机和平板电脑）。不像封闭系统（例如，自动驾驶系统），其中预先知道一组共同运行的应用程序，开放式移动的系统的用户可以在任何时间安装或卸载任意应用程序，并且集中式解决方案受到采用障碍的影响。这项工作提出了第一个已知的分散的应用程序级调度机制来解决这个问题。通过利用深度强化学习的自适应性，该解决方案可以使合作运行的应用程序的调度收敛到纳什均衡点，从而在应用程序之间产生良好的收益平衡。该解决方案还可以自动适应运行环境以及底层操作系统和硬件。实验表明，该解决方案在DNN工作负载、硬件配置和运行场景中持续产生显著的加速和节能效果。
As more apps embrace AI, it is becoming increasingly common that multiple Deep Neural Networks (DNN)-powered apps may run at the same time on a mobile device. This paper explores scheduling in such multi-instance DNN scenarios, on general open mobile systems (e.g., common smartphones and tablets). Unlike closed systems (e.g., autonomous driving systems) where the set of co-run apps are known beforehand, the user of an open mobile system may install or uninstall arbitrary apps at any time, and a centralized solution is subject to adoption barriers. This work proposes the first-known decentralized application-level scheduling mechanism to address the problem. By leveraging the adaptivity of Deep Reinforcement Learning, the solution is shown to make the scheduling of co-run apps converge to a Nash equilibrium point, yielding a good balance of gains among the apps. The solution more-over automatically adapts to the running environment and the underlying OS and hardware. Experiments show that the so-lution consistently produces significant speedups and energy savings across DNN workloads, hardware configurations, and running scenarios.