DQN-based QoE Enhancement for Data Collection in Heterogeneous IoT Network
DQN-based QoE Enhancement for Data Collection in Heterogeneous IoT Network
复制标题
DOI：
10.1109/mass56207.2022.00032
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Hansong Zhou;Sihan Yu;Xiaonan Zhang;Linke Guo;B. Lorenzo
中科院分区：
文献类型：
--
作者：
Hansong Zhou;Sihan Yu;Xiaonan Zhang;Linke Guo;B. Lorenzo
物联网（IoT）设备的传感数据采集为支持大规模物联网应用奠定了基础，例如智能健康中的患者监测和智能制造中的智能控制。不幸的是，物联网设备和动态环境的异质性不仅会导致生命周期延迟，还会导致数据收集失败，从而影响所有用户的体验质量（QoE）。在本文中，我们提出了一种采用动态数据污染方法的恢复机制来处理故障。为了进一步提高长期整体QoE，我们使用深度强化学习方法分配频谱资源并为每个设备做出污染决策。特别地，提出了一种轻量级的分散状态共享深度循环q网络（SDRQN）来寻找最优的收集策略。仿真结果表明，与全连接设计相比，SDRQN中的循环单元可减少10%的等待时间和60%的任务掉落率。与集中式DQN方案相比，SDRQN实现了0.29%的超低丢料率，但只需要1%的GPU内存，证明了SDRQN在大规模异构物联网网络中的有效性。
Sensing data collection from the Internet of Things (IoT) devices lays the foundation to support massive IoT applications, such as patient monitoring in smart health and intelligent control in smart manufacturing. Unfortunately, the heterogeneity of IoT devices and dynamic environments result in not only the life-cycle latency but also data collection failures, affecting the quality of experience (QoE) for all the users. In this paper, we propose a recovery mechanism with a dynamic data contamination method to handle the failure. To further enhance the long-term overall QoE, we allocate the spectrum resources and make contamination decisions for each device using a deep reinforcement learning method. Particularly, a lightweight decentralized State-sharing Deep-Recurrent Q-Network (SDRQN) is proposed to find the optimal collection policies. Our simulation results indicate that the recurrent unit in SDRQN gives rise to 10% lower waiting time and 60% lower task drop rate than the fully-connected design. Compared to a centralized DQN scheme, SDRQN achieves a similar ultra-low drop rate of 0.29% but requires only 1% GPU memory, demonstrating the effectiveness of SDRQN in the large-scale heterogeneous IoT network.