Asynchronous Federated Deep Reinforcement Learning-Based URLLC-Aware Computation Offloading in Space-Assisted Vehicular Networks
Asynchronous Federated Deep Reinforcement Learning-Based URLLC-Aware Computation Offloading in Space-Assisted Vehicular Networks
复制标题
空间辅助车载网络中基于异步联邦深度强化学习的URLLC感知计算卸载
DOI：
10.1109/tits.2022.3150756
复制
发表时间：
2022-02-24
影响因子：
8.5
通讯作者：
Al-Otaibi, Sattam
中科院分区：
文献类型：
--
作者：
Pan, Chao;Wang, Zhao;Al-Otaibi, Sattam
空间辅助车辆网络（SAVN）为用户车辆（UV）提供无缝覆盖和按需数据处理服务。然而，现有的计算卸载技术很难在 SAVN 中满足新兴车辆应用提出的超可靠和低延迟通信 (URLLC) 需求。由于环境观测的利用不足，传统的深度强化学习算法不适合高度动态的 SAVN。本文提出了一种基于异步联邦深度 Q 学习 (DQN) 和 URLLC 感知的计算卸载算法 (ASTEROID)，以在考虑长期 URLLC 约束的情况下实现吞吐量最大化。具体来说，我们首先建立基于极值理论的URLLC约束模型。其次，采用李亚普诺夫优化来分解任务卸载和计算资源分配。最后，提出了一种基于异步联邦DQN（AF-DQN）算法来解决UV端任务卸载问题。服务器端计算资源分配通过队列积压感知算法来解决。仿真结果验证了 ASTEROID 实现了卓越的吞吐量和 URLLC 性能。
Space-assisted vehicular networks (SAVN) provide seamless coverage and on-demand data processing services for user vehicles (UVs). However, ultra-reliable and low-latency communication (URLLC) demands imposed by emerging vehicular applications are hard to be satisfied in SAVN by existing computation offloading techniques. Traditional deep reinforcement learning algorithms are unsuitable for highly dynamic SAVN due to the underutilization of environment observations. An AsynchronouS federaTed deep Q-learning (DQN)-basEd and URLLC-aware cOmputatIon offloaDing algorithm (ASTEROID) is presented in this paper to achieve throughput maximization considering the long-term URLLC constraints. Specifically, we first establish an extreme value theory-based URLLC constraint model. Second, the task offloading and computation resource allocation are decomposed by employing Lyapunov optimization. Finally, an asynchronous federated DQN-based (AF-DQN) algorithm is presented to address the UV-side task offloading problem. The server-side computation resource allocation is settled by an queue backlog-aware algorithm. Simulation results verify that ASTEROID achieves superior throughput and URLLC performances.