Deep Reinforcement Learning for AoI Aware VNF Placement in Multiple Source Systems
Deep Reinforcement Learning for AoI Aware VNF Placement in Multiple Source Systems
复制标题
DOI：
10.1109/globecom48099.2022.10001066
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Zhenke Chen;He Li;K. Ota;M. Dong
中科院分区：
文献类型：
--
作者：
Zhenke Chen;He Li;K. Ota;M. Dong
信息时代（AoI）是一种新出现的绩效指标，用于从目的地的角度量化数据的新鲜度。在本文中，我们在多源更新系统的背景下调查和分析 AoI。在这样的系统中，多个 IoT 设备持续监控物理环境，并通过支持网络功能虚拟化 (NFV) 的网络将数据发送到远程目的地以进行状态更新。考虑到虚拟网络功能（VNF）放置可能不必要地影响更新的AoI，我们研究了此类系统中的VNF放置问题。因此，该问题被表述为数学优化问题，旨在最小化在目的地接收的所有更新的长期平均 AoI。为了解决这个问题，我们提出了一种基于深度强化学习（DRL）的 VNF 放置方法，称为 VNF-AoI，其中学习代理或决策者与系统环境交互，从而根据其学到的经验提供最佳的 VNF 放置策略。最后，我们进行了广泛的模拟来验证我们提出的方法的有效性。数值结果清楚地表明，我们的 VNF-AoI 优于其他两种基线算法，目的地的接受率平均高出 13.8%，平均 AoI 低 20.3%。
Age of Information (AoI) is a newly emergent performance metric to quantify the freshness of data from destinations' perspectives. In this paper, we investigate and analyze AoI in the context of a multiple source updating system. In such a system, multiple loT devices continuously monitors physical environment and sends data to a remote destination for status updates through a Network Function Virtualization (NFV)-enabled network. Considering that the Virtual Network Function (VNF) placement can unnecessarily influence the AoI of the updates, we study the VNF placement problem in such a system. The problem is hence formulated as a mathematical optimization problem aiming to minimize the long-term average AoI of all updates received at the destination. To solve this prob-lem, we propose a Deep Reinforcement Learning (DRL)-based VNF placement approach called VNF-AoI, where a learning agent or decision-maker interacts with a system environment and consequently provides an optimal VNF placement policy according to the experience it has learned. Finally, we conduct extensive simulations to validate the effectiveness of our proposed approach. Numerical results clearly demonstrate that our VNF-AoI surpasses other two baseline algorithms by averagely 13.8 % higher acceptance ratio and 20.3 % lower average AoI at the destination.