Automated Ensemble for Deep Learning Inference on Edge Computing Platforms
Automated Ensemble for Deep Learning Inference on Edge Computing Platforms
复制标题
边缘计算平台上深度学习推理的自动集成
DOI：
10.1109/jiot.2021.3102945
复制
发表时间：
2022-03
影响因子：
10.6
通讯作者：
Yang Bai;Lixing Chen;M. Abdel-Mottaleb;Jie Xu
中科院分区：
文献类型：
--
作者：
Yang Bai;Lixing Chen;M. Abdel-Mottaleb;Jie Xu
深度学习（DL）的进步引发了移动的智能的爆炸式增长，对计算资源的需求飙升，这是移动的设备无法满足的。在本文中，我们使用边缘计算为最终用户提供更好的DL推理服务。关键是利用深度神经网络（DNN）集成技术，为许多机器学习应用程序在推理准确性和鲁棒性方面提供最先进的性能。与终端设备相比，边缘计算平台被赋予了更强大的计算资源，使得实现深度学习推理的DNN集成成为可能。然而，由于边缘服务器的有限计算能力和可能的服务响应截止日期，边缘服务器只能使用有限数量的DNN来构建DNN集成。这提出了一个独特的问题，即DNN集成选择，用于识别最适合的DNN集成。我们提出了一种新的算法称为自动DNN集成选择（AES）算法来解决这个问题。由于DNN在不同的输入数据分布上表现出性能差异，AES根据所接纳的推理任务的特征自适应地确定DNN集成。AES是一种在线学习算法，可以随着时间的推移学习DNN的使用性能。集成选择规则被进一步设计为AES的子程序，以基于DNN的准确性和多样性来招募成员到DNN集成。特别地，我们从理论上证明了AES可以达到渐近最优。我们在真实世界的数据集上进行实验。结果表明，在边缘计算平台上使用DNN集成技术显著提高了DL推理质量，AES优于其他基准方案。
Advances in deep learning (DL) have triggered an explosion of mobile intelligence, posing a soaring demand for computing resources that cannot be satisfied by mobile devices. In this article, we employ edge computing to deliver better DL inference services to end users. The key is to leverage deep neural network (DNN) ensemble techniques that provide state-of-the-art performance for many machine learning applications in terms of inference accuracy and robustness. Compared to end devices, the edge computing platform is endowed with more powerful computing resources, making it feasible to implement DNN ensembles for DL inferences. However, due to the constrained computing capacity of edge servers and the possible service response deadline, an edge server can only use a limited number of DNNs to construct DNN ensembles. This poses a unique problem, namely, DNN ensemble selection, for identifying the best-fit DNN ensembles. We propose a novel algorithm called automated DNN ensemble selection (AES) algorithm to solve this problem. Because DNNs exhibit performance variations over different distributions of input data, AES adaptively determines a DNN ensemble according to the features of admitted inference tasks. AES is an online learning algorithm that learns DNNs’ in-use performance over time. An ensemble selection rule is further designed as a subroutine of AES to recruit members into the DNN ensemble based on the accuracy and diversity of DNNs. In particular, we theoretically prove that AES can achieve asymptotic optimality. We carry out experiments on real-world data sets. The results show that using the DNN ensemble technique on edge computing platforms dramatically improves the DL inference quality, and AES outperforms other benchmark schemes.