Power Management of Wireless Sensor Nodes with Coordinated Distributed Reinforcement Learning
Power Management of Wireless Sensor Nodes with Coordinated Distributed Reinforcement Learning
复制标题
具有协调分布式强化学习的无线传感器节点的电源管理
DOI：
10.1109/iccd46524.2019.00092
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Nakamura Hiroshi
中科院分区：
文献类型：
--
作者：
Shresthamali Shaswot;Kondo Masaaki;Nakamura Hiroshi
能量收集无线传感器节点（EHWSNs）需要自适应的能量管理策略，以便在其物理环境中不间断地永久运行。当代在线强化学习（RL）解决方案需要花费不切实际的长时间来探索环境，以收敛到工作策略。我们的工作通过划分多个代理同时探索的状态空间来加速学习。我们实现了这一点，通过使用一种新的协调e-greedy方法，并通过分布式RL（DiRL）在EHWSN网络中实现它。我们的模拟结果显示，状态空间渗透率增加了四倍，时间减少了一个数量级（50倍），以实现最佳操作。此外，我们还提出了减少与学习和探索相关的灾难性结果的方法。这意味着将与真实场景对应的模拟中的节点停机时间减少了三分之一。
Energy Harvesting Wireless Sensor Nodes (EHWSNs) require adaptive energy management policies for uninterrupted perpetual operation in their physical environments. Contemporary online Reinforcement Learning (RL) solutions take an unrealistically long time exploring the environment to converge on working policies. Our work accelerates learning by partitioning the state-space for simultaneous exploration by multiple agents. We achieve this by using a novel coordinated e-greedy method and implement it via Distributed RL (DiRL) in an EHWSN network. Our simulation results show a four-fold increase in state-space penetration and reduction in time to achieve optimal operation by an order of magnitude (50x). Moreover, we also propose methods to reduce instances of disastrous outcomes associated with learning and exploration. This translates to reducing the downtimes of the nodes in simulations corresponding to a real-world scenario by one thirds.