Learning to Transmit Fresh Information in Energy Harvesting Networks
Learning to Transmit Fresh Information in Energy Harvesting Networks
复制标题
DOI：
10.1109/tgcn.2022.3190007
复制
发表时间：
2022-12
影响因子：
4.8
通讯作者：
Shiyang Leng;A. Yener
中科院分区：
文献类型：
--
作者：
Shiyang Leng;A. Yener
我们研究了一个由能量收集发射器组成的自组织网络中的信息年龄 (AoI) 最小化，这些发射器计划向其预期接收器发送状态更新。首先研究通信会话上具有功率分配问题的传输调度，假设先验知识信道状态信息、收获的能量和更新分组到达，即离线设置。在这种情况下，全局最优调度策略是已知计算困难的混合整数线性规划的解决方案。我们提出了一种基于监督学习的算法来减轻高计算复杂性。将用户调度解释为时间序列分类问题的双向循环神经网络经过训练和测试，以实现接近最优的 AoI。接下来，我们考虑具有系统状态因果知识的在线调度和功率分配，这是一个无限状态马尔可夫决策问题。在这种情况下，相关的强化学习问题通过无模型的同策略深度强化学习来解决，其中实现了具有深度神经网络函数逼近的行动者批评算法。展示了与最优的可比较的 AoI，并观察到学习求解器的更快运行时间，验证了无线网络中以 AoI 为中心的调度和资源分配问题的最优性和计算能效方面的学习效果。
We study age of information (AoI) minimization in an ad hoc network consisting of energy harvesting transmitters that are scheduled to send status updates to their intended receivers. The transmission scheduling with power allocation problem over a communication session is first studied assuming apriori knowledge of channel state information, harvested energy, and update packet arrivals, i.e., the offline setting. The global optimal scheduling policy in this case is the solution of a mixed integer linear program which is known to be computationally hard. We propose a supervised-learning-based algorithm to mitigate the high computational complexity. A bidirectional recurrent neural network that interprets user scheduling as a time-series classification problem is trained and tested to achieve near-optimal AoI. Next, we consider online scheduling and power allocation with causal knowledge of the system state, which is an infinite-state Markov decision problem. In this case, the related reinforcement learning problem is solved by a model-free on-policy deep reinforcement learning, where the actor-critic algorithm with deep neural network function approximation is implemented. Comparable AoI to the optimal is demonstrated and faster runtime of learning solvers is observed, verifying the efficacy of learning in terms of both optimality and computational energy efficiency for AoI-focused scheduling and resource allocation problems in wireless networks.