Deep Reinforcement Learning with IoT System Characterization and Knowledge Adaptation
Deep Reinforcement Learning with IoT System Characterization and Knowledge Adaptation
复制标题
DOI：
10.1109/uemcon54665.2022.9965641
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Jiadao Zou;Qingxue Zhang
中科院分区：
文献类型：
--
作者：
Jiadao Zou;Qingxue Zhang
由于大数据系统中有许多设备，因此如何根据系统约束（如能源预算）配置设备是一个挑战。典型的场景包括许多可穿戴健康传感器或物联网。在这项研究中，针对这一挑战，我们提出了一种新的深度强化学习框架，用于配置系统以实现最佳操作，该框架通过动作和反馈自动学习。我们的新贡献是双重的，这大大提高了性能。首先，我们提出了一种系统特征化的方法，可以提取许多设备的系统状态的模式，以确定系统状态是否有显着的变化。其次，我们提出了一个知识适应的方法来确定何时更新学习缓冲区，以及如何选择一个学习批次。此外，我们还研究了超参数，包括学习率，批量归一化和正则化方法，对深度强化学习结果的影响。在多设备系统设置上进行评估，所提出的框架已经证明了新颖设计的显着性能提升。这项研究将大大推进具有许多可穿戴设备或物联网设备的系统的智能配置，以实现大数据实践。
With many devices in the big data system, the challenge arises on how to configure the devices based on the system constraints like the energy budget. The typical scenarios include many wearable health sensors or inter-of-things. In this study, targeting this challenge, we propose a novel deep reinforcement learning framework for configuring the system for optimal operations, which automatically learns through actions and feedbacks. Our novel contributions are two-fold, which greatly boost the performance. First, we propose a system characterization approach that can extract the patterns of the system states of many devices to determine whether the system state has significant changes. Second, we propose a knowledge adaptation approach to determine when to update the learning buffer and how to select a learning batch. Further, we have investigated hyperparameters including learning rate, batch normalization and regularization methods, on deep reinforcement learning outcomes. Evaluated on a multi-device system setup, the proposed framework has demonstrated significant performance boosting with the novel designs. This study will greatly advance intelligent configuration for systems with many wearables or IoT devices, towards big data practices.