Joint Edge Computing and Caching Based on D3QN for the Internet of Vehicles
Joint Edge Computing and Caching Based on D3QN for the Internet of Vehicles
复制标题
DOI：
10.3390/electronics12102311
复制
发表时间：
2023-05
期刊：
影响因子：
2.9
通讯作者：
Geng Chen;J. Sun;Qingtian Zeng;Gang Jing;Yudong Zhang
中科院分区：
文献类型：
--
作者：
Geng Chen;J. Sun;Qingtian Zeng;Gang Jing;Yudong Zhang
随着车联网（IOV）的发展，许多自动驾驶汽车（SDV）需要处理各种任务，但计算和存储资源非常有限，这意味着它们无法及时完成密集的任务。针对多任务联合边计算和缓存过程中存在的问题，提出了一种基于决斗双深Q网络（D3 QN）的联合边计算和缓存方法。首先，卸载任务和缓存到基站的过程被建模为优化问题，以最大限度地提高系统收益，这是有限的系统延迟和能量消耗以及缓存空间的计算任务的约束。此外，我们还考虑了未完成任务的数量对优化问题的负面影响-未完成任务的数量越高，系统收益越低。其次，我们使用的D3 QN算法与该高速缓存模型来解决所制定的NP-难问题，并选择最佳的缓存和卸载行动，采用e-greedy策略。此外，本文提出了两种任务缓存模型，即基于任务流行度的主动缓存和基于D3 QN算法的被动缓存。此外，处理高速缓存空间的任务通过基于流行度类型计算驱逐值来更新。仿真结果表明，该算法在系统延迟和能耗方面具有良好的性能，提高了缓存空间利用率，降低了任务未完成的概率。与采用缓存策略的Deep Q网络相比，采用缓存策略的Double Deep Q网络和采用缓存策略的Dueling Deep Q网络的系统收益分别提高了65%、35%和66%。本文提出的物联网场景可以通过增加SDV和基站的数量扩展到更大规模的物联网系统，物联网的内容缓存和下载功能也可以通过多个基站之间的协作来实现。但本文只关注了该高速缓存模型，替换模型的设计不够好，导致缓存资源利用率较低。在未来的工作中，我们将分析如何在具有多个异构服务的IOV场景中基于多代理协作进行缓存，卸载和替换的联合决策，以支持不同的Vehicle-to-Everything服务。
With the Internet of Vehicles (IOV), a lot of self-driving vehicles (SDVs) need to handle a variety of tasks but have very seriously limited computing and storage resources, meaning they cannot complete intensive tasks timely. In this paper, a joint edge computing and caching based on a Dueling Double Deep Q Network (D3QN) is proposed to solve the problem of the multi-task joint edge calculation and caching process. Firstly, the processes of offloading tasks and caching them to the base station are modeled as optimization problems to maximize system revenues, which are limited by system latency and energy consumption as well as cache space for computing task constraints. Moreover, we also take into account the negative impact of the number of unfinished tasks in relation to the optimization problem—the higher the number of unfinished tasks, the lower the system revenue. Secondly, we use the D3QN algorithm together with the cache models to solve the formulated NP-hard problem and select the optimal caching and offloading action by adopting an e-greedy strategy. Moreover, two cache models are proposed in this paper to cache tasks, namely the active cache, based on the popularity of the task, and passive cache, based on the D3QN algorithm. Additionally, tasks which deal with cache space are updated by computing the expulsion value based on type of popularity. Finally, simulation results show that the proposed algorithm has good performance in terms of the latency and energy consumption of the system and that it improves utilization of cache space and reduces the probability of unfinished tasks. Compared to the Deep Q Network with caching policy, with the Double Deep Q Network with caching policy and Dueling Deep Q Network with caching policy, the system revenue of the proposed algorithm is improved by 65%, 35% and 66%, respectively. The scenario of the IOV proposed in this article can be expanded to larger-scale IOV systems by increasing the number of SDVs and base stations, and the content caching and download functions of the Internet of Things can also be achieved through collaboration between multiple base stations. However, only the cache model is focused on in this article, and the design of the replacement model is not good enough, resulting in a low utilization of cache resources. In future work, we will analyze how to make joint decisions based on multi-agent collaboration for caching, offloading and replacement in IOV scenarios with multiple heterogeneous services to support different Vehicle-to-Everything services.