Mitigation of Scheduling Violations in Time-Sensitive Networking using Deep Deterministic Policy Gradient
Mitigation of Scheduling Violations in Time-Sensitive Networking using Deep Deterministic Policy Gradient
复制标题
使用深度确定性策略梯度缓解时间敏感网络中的调度违规
DOI：
10.1145/3472735.3473385
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Cheng, Liang
中科院分区：
文献类型：
--
作者：
Zhou, Boyang;Cheng, Liang
时间敏感网络 (TSN) 专为实时应用程序而设计，通常与一组时间触发 (TT) 数据流相关。 TT 流量通常要求低丢包率并保证端到端延迟上限。为了保证端到端延迟范围，TSN 使用时间感知整形器 (TAS) 为 TT 流提供确定性服务。 TT 流量的每个帧都在每个交换机上安排一个特定的时隙进行传输。有几个因素可能会影响帧的传输，从而影响整个网络的调度。这些因素都可能导致帧在错误的时隙中发送，即错误行为。为了减少不当行为的发生，我们需要为整个网络找到适当的调度。在我们的研究中，我们使用强化学习模型（称为深度确定性策略梯度（DDPG））来找到合适的调度。 DDPG 用于对时间同步误差等传输影响因素引起的不确定性进行建模。与现有技术相比，我们使用 DDPG 的方法显着减少了所研究的 TSN 场景中的不当行为数量，并提高了网络的延迟性能。
Time-Sensitive Networking (TSN) is designed for real-time applications, usually pertaining to a set of Time-Triggered (TT) data flows. TT traffic generally requires low packet loss and guaranteed upper bounds on end-to-end delay. To guarantee the end-to-end delay bounds, TSN uses Time-Aware Shaper (TAS) to provide deterministic service to TT flows. Each frame of TT traffic is scheduled a specific time slot at each switch for its transmission. Several factors may influence frame transmissions, which then impact the scheduling in the whole network. These factors may cause frames sent in wrong time slots, namely misbehaviors. To mitigate the occurrence of misbehaviors, we need to find proper scheduling for the whole network. In our research, we use a reinforcement-learning model, which is called Deep Deterministic Policy Gradient (DDPG), to find the suitable scheduling. DDPG is used to model the uncertainty caused by the transmission-influencing factors such as time-synchronization errors. Compared with the state of the art, our approach using DDPG significantly decreases the number of misbehaviors in TSN scenarios studied and improves the delay performance of the network.