Enhanced Online Q-Learning Scheme for Resource Allocation with Maximum Utility and Fairness in Edge-IoT Networks
Enhanced Online Q-Learning Scheme for Resource Allocation with Maximum Utility and Fairness in Edge-IoT Networks
复制标题
DOI：
10.1109/tnse.2020.3015689
复制
发表时间：
2020-10-01
影响因子：
6.6
通讯作者：
Pan, Jianli
中科院分区：
文献类型：
--
作者：
AlQerm, Ismail;Pan, Jianli
物联网（IoT）由于异构应用数量的增加而正在经历数据流量的爆炸式增长。现有的云计算模型将无法支持延迟敏感和使用高带宽的物联网应用程序。以共享边缘云为代表的边缘物联网系统支持广泛的物联网应用。边缘云提供更接近物联网设备的资源，以解决延迟敏感性和带宽问题。然而，在具有多个异构物联网应用、各种资源需求和有限资源可用性的边缘物联网的上下文中，在保证应用效用的情况下分配这些资源是具有挑战性的。在本文中，我们提出了一种新的增强型在线Q学习方案，用于将资源从边缘云分配到物联网应用程序，以最大化其效用并保持分配公平性。所开发的在线Q学习方案近似其Q值，以解决大的状态空间的问题，减少所需的学习计算，加快系统的收敛。它使用两种设置来实现：使用边缘云上的专用控制器进行集中式，以及边缘服务器进行分布式学习，以实现找到联合资源分配策略的共同目标，从而最大限度地提高物联网应用程序的效用。大量的数值结果表明，该方案在提高应用程序的效用和分配公平性的能力。
Internet of Things (IoT) is experiencing an explosion in the data traffic due to the increase in the number of heterogeneous applications. The existing cloud computing models will not be capable to support the IoT applications that are delay-sensitive and using high bandwidth. The Edge-IoT systems represented by shared edge clouds support a wide range of IoT applications. Edge clouds provide resources closer to the IoT devices to tackle the delay sensitivity and bandwidth issues. However, the allocation of these resources with guaranteed application's utility in the context of Edge-IoT with multiple heterogeneous IoT applications, various resource demands, and limited resource availability is challenging. In this paper, we propose a novel enhanced online Q-learning scheme to allocate resources from edge clouds to IoT applications to maximize their utility and maintain allocation fairness among them. The developed online Q-learning scheme approximates its Q-value to tackle the problem of large state space, reduce the required learning computation, and expedite the system convergence. It is implemented using two settings: centralized using a dedicated controller at the edge cloud and distributed where edge servers learn cooperatively to achieve a common goal of finding joint resource allocation policy that maximizes the IoT applications' utilities. Extensive numerical results demonstrate the capability of the proposed scheme in improving applications' utilities and allocation fairness.