Learning to Sample a Signal through an Unknown System for Minimum AoI
Learning to Sample a Signal through an Unknown System for Minimum AoI
复制标题
学习通过未知系统对信号进行采样以获得最小 AoI
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
A. Ephremides
中科院分区：
文献类型：
--
作者：
C. Kam;S. Kompella;A. Ephremides
在这项工作中，我们通过优化通过具有未知延迟轮廓的系统传输信号的采样策略来考虑最小化信息年龄（AOI）的问题。我们的目标是观察接收者的年龄并在线学习如何采样以最大程度地减少平均年龄。我们开始使用一个具有未知服务时间分配的无损单人服务器系统的特殊情况，并在收到每个数据包后决定等待多长时间，然后再进行采样和传输新数据包。我们得出了指数服务器的最佳阈值策略，该服务器能够通过估计服务率来在线学习最佳策略，甚至在服务率突然变化时甚至可以适应。但是，我们表明，当分布不是指数级时，这种方法可能会失败，因此我们考虑不依赖指数服务器假设的强化学习（RL）方法（带瓷砖编码的SARSA）。我们将RL方法适应AOI最小化问题，并表明在分布不是指数的情况下，它的表现优于其他估计的阈值策略。最后，我们讨论了不仅假设单个服务器的问题的扩展，并提出了一些方法来解决未知系统采样的更一般问题。
In this work, we consider the problem of minimizing age of information (AoI) by optimizing the sampling strategy for transmitting a signal through a system with an unknown delay profile. Our goal is to observe the age at the receiver and learn online how to sample to minimize the average age. We begin our investigation with a special case of a lossless single-server system with unknown service time distribution, and decide after each packet is received how long to wait before sampling and transmitting a new packet. We derive the optimal threshold policy for an exponential server, which is able to learn the optimal policy online by estimating the service rate, and is even able to adapt when there is an abrupt change in the service rate. However, we show that this approach can fail when the distribution is not exponential, so we consider a reinforcement learning (RL) approach (Sarsa with tile coding) that does not rely on the exponential server assumption. We adapt the RL approach to the AoI minimization problem, and show that it outperforms the other estimated threshold policy in the case where the distribution is not exponential. Lastly, we discuss extensions to the problem that do not assume just a single server, and propose some approaches to solve the more general problem of sampling for an unknown system.