Toward Reliability-Enhanced, Delay-Guaranteed Dynamic Network Slicing: A Multiagent DQN Approach With an Action Space Reduction Strategy
Toward Reliability-Enhanced, Delay-Guaranteed Dynamic Network Slicing: A Multiagent DQN Approach With an Action Space Reduction Strategy
复制标题
DOI：
10.1109/jiot.2023.3323817
复制
发表时间：
2024-03
影响因子：
10.6
通讯作者：
Weili Wang;Lun Tang;Tong Liu;Xiaoqiang He;Chengchao Liang;Qianbin Chen
中科院分区：
文献类型：
--
作者：
Weili Wang;Lun Tang;Tong Liu;Xiaoqiang He;Chengchao Liang;Qianbin Chen
网络可用性和服务连续性是网络运营商为物联网(IoT)提供可靠通信服务的主要问题，在网络服务同时面临软件(虚拟网络功能(VNF)实例)和硬件(物理节点)故障风险的虚拟网络切片环境中，实现这一点尤其具有挑战性。通常采用基于冗余的VNF备份方案来提高虚拟网片的可靠性。但是，备份VNF需要的资源量与主VNF相同，这将导致较高的资源成本。在本文中，我们提出了一种用于VNF编排、备份和映射的联合VNF分区和混合备份方案，其目的是以最小的代价构建可靠性增强和延迟保证的网络切片。具体地说，VNF划分方法将单个VNF划分为多个处理能力较低的更薄的VNF，有望以更少的额外资源提高网络切片的可靠性。混合备份方案包括现场和异地备份形式。然后，考虑到时变的网络环境和物联网服务需求，将VNF编排、备份和映射问题描述为动态混合整数线性规划(DMILP)问题，并将动态问题建模为马尔可夫决策过程(MDP)。针对MDP的动作空间较大的特点，提出了一种基于动作空间约简的多智能体深度强化学习(DRL)方法，实现了VNF的动态编排、备份和映射。仿真结果表明，该混合备份方案能够以较低的网络开销获得较好的时延和可靠性性能。
Network availability and service continuity are major concerns for network operators to provide reliable communication services for Internet of Things (IoT), which are particularly challenging to achieve in virtualized network slicing environment where network services are exposed to the failure risks of both software (virtual network function (VNF) instances) and hardware (physical nodes). In general, the redundancy-based VNF backup solutions are used to improve the reliability of virtualized network slices. However, backup VNFs require the same amount of resources as the primary VNFs, which will result in high-resource cost. In this article, we propose a joint VNF partition and hybrid backup scheme for VNF orchestration, backup and mapping, whose aim is to construct the reliability-enhanced and delay-guaranteed network slices at minimum cost. Specifically, the VNF partition method divides a single VNF into multiple thinner VNFs with lower processing capacity and is expected to enhance the reliability of network slices with less additional resources. The hybrid backup scheme includes both onsite and offsite backup forms. Then, considering the time-varying network environment and IoT service requirements, we formulate the VNF orchestration, backup and mapping as a dynamic mixed integer linear programming (DMILP) problem, and model the dynamic problem as a Markov decision process (MDP). In view of the large action space of the formulated MDP, we propose a multiagent deep reinforcement learning (DRL) approach with an action space reduction strategy to achieve the dynamic VNF orchestration, backup and mapping solution. Simulation results demonstrate that the proposed joint VNF partition and hybrid backup scheme can obtain superior delay and reliability performance with low-network cost.