DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications
DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications
复制标题
DOI：
10.1109/tnsm.2021.3123959
复制
发表时间：
2021-12-01
影响因子：
5.3
通讯作者：
Pan, Jianli
中科院分区：
文献类型：
--
作者：
AlQerm, Ismail;Pan, Jianli
边缘计算正在兴起，为物联网（IoT）应用的未来赋能。然而，由于应用的异构性，有效分配多维有限资源（CPU、内存、存储、带宽等）是边缘云的重大挑战。具有应用的服务质量（QoS）要求的约束。在本文中，我们通过开发一个名为DeepEdge的新框架来解决边缘物联网系统中的资源分配问题，该框架将资源分配给异构物联网应用程序，目标是最大限度地提高用户的体验质量（QoE）。为了实现这一目标，我们开发了一种新的QoE模型，该模型考虑将物联网应用的异构需求与可用的边缘资源相匹配。通过选择可用资源可以满足的QoS要求范围来实现对齐。此外，我们提出了一种新的两阶段深度强化学习（DRL）方案，该方案有效地分配边缘资源以服务于物联网应用并最大化用户的QoE。与典型的DRL不同，我们的方案利用深度神经网络（DNN）来改进动作的探索，通过使用DNN将边缘物联网状态映射到由资源分配和QoS类组成的联合资源分配动作。这种联合行动不仅最大化了用户的QoE，满足了异构应用的需求，而且使QoS需求与可用资源相匹配。此外，我们开发了一种Q值近似方法来解决边缘物联网的大空间问题。进一步的评估表明，与现有的资源分配方案相比，DeepEdge在QoE，延迟和应用任务的成功率方面带来了相当大的改善。
Edge computing is emerging to empower the future of Internet of Things (IoT) applications. However, due to heterogeneity of applications, it is a significant challenge for the edge cloud to effectively allocate multidimensional limited resources (CPU, memory, storage, bandwidth, etc.) with constraints of applications' Quality of Service (QoS) requirements. In this paper, we address the resource allocation problem in Edge-IoT systems through developing a novel framework named DeepEdge that allocates resources to the heterogeneous IoT applications with the goal of maximizing users' Quality of Experience (QoE). To achieve this goal, we develop a novel QoE model that considers aligning the heterogeneous requirements of IoT applications to the available edge resources. The alignment is achieved through selection of QoS requirement range that can be satisfied by the available resources. In addition, we propose a novel two-stage deep reinforcement learning (DRL) scheme that effectively allocates edge resources to serve the IoT applications and maximize the users' QoE. Unlike the typical DRL, our scheme exploits deep neural networks (DNN) to improve actions' exploration by using DNN to map the Edge-IoT state to joint resource allocation action that consists of resource allocation and QoS class. The joint action not only maximize users' QoE and satisfies heterogeneous applications' requirements but also align the QoS requirements to the available resources. In addition, we develop a Q-value approximation approach to tackle the large space problem of Edge-IoT. Further evaluation shows that DeepEdge brings considerable improvements in terms of QoE, latency and application tasks' success ratio in comparison to the existing resource allocation schemes.