Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments Using A3C Learning and Residual Recurrent Neural Networks
Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments Using A3C Learning and Residual Recurrent Neural Networks
复制标题
DOI：
10.1109/tmc.2020.3017079
复制
发表时间：
2022-03-01
影响因子：
7.9
通讯作者：
Buyya, Rajkumar
中科院分区：
文献类型：
--
作者：
Tuli, Shreshth;Ilager, Shashikant;Buyya, Rajkumar
基于物联网 (IoT) 的应用程序的普遍采用导致了雾计算范式的出现，它允许无缝地利用移动边缘和云资源。由于资源能力有限、物联网中的移动性因素、资源异构性、网络层次结构和随机行为，在此类环境中高效调度应用程序任务具有挑战性。现有的基于启发式和强化学习的方法缺乏通用性和快速适应性，因此无法最佳地解决这个问题。它们也无法利用临时工作负载模式，并且仅适用于集中式设置。然而，众所周知，异步优势演员批评家（A3C）学习可以快速适应动态场景，数据较少，残差循环神经网络（R2N2）可以快速更新模型参数。因此，我们提出了一种基于 A3C 的实时调度程序，用于随机边缘云环境，允许跨多个代理同时进行去中心化学习。我们使用 R2N2 架构来捕获大量主机和任务参数以及时间模式，以提供高效的调度决策。所提出的模型是自适应的，能够根据应用需求调整不同的超参数。我们通过敏感性分析来解释我们对超参数的选择。在真实数据集上进行的实验表明，与最先进的算法相比，在能耗、响应时间、服务级别协议和运行成本方面分别显着改善了 14.4%、7.74%、31.9% 和 4.64%。
The ubiquitous adoption of Internet-of-Things (IoT) based applications has resulted in the emergence of the Fog computing paradigm, which allows seamlessly harnessing both mobile-edge and cloud resources. Efficient scheduling of application tasks in such environments is challenging due to constrained resource capabilities, mobility factors in IoT, resource heterogeneity, network hierarchy, and stochastic behaviors. Existing heuristics and Reinforcement Learning based approaches lack generalizability and quick adaptability, thus failing to tackle this problem optimally. They are also unable to utilize the temporal workload patterns and are suitable only for centralized setups. However, asynchronous-advantage-actor-critic (A3C) learning is known to quickly adapt to dynamic scenarios with less data and residual recurrent neural network (R2N2) to quickly update model parameters. Thus, we propose an A3C based real-time scheduler for stochastic Edge-Cloud environments allowing decentralized learning, concurrently across multiple agents. We use the R2N2 architecture to capture a large number of host and task parameters together with temporal patterns to provide efficient scheduling decisions. The proposed model is adaptive and able to tune different hyper-parameters based on the application requirements. We explicate our choice of hyper-parameters through sensitivity analysis. The experiments conducted on real-world data set show a significant improvement in terms of energy consumption, response time, Service-Level-Agreement and running cost by 14.4, 7.74, 31.9, and 4.64 percent, respectively when compared to the state-of-the-art algorithms.