Symbolic Distillation for Learned TCP Congestion Control
Symbolic Distillation for Learned TCP Congestion Control
复制标题
DOI：
10.48550/arxiv.2210.16987
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
S. Sharan;Wenqing Zheng;Kuo-Feng Hsu;Jiarong Xing;Ang Chen;Zhangyang Wang
中科院分区：
文献类型：
--
作者：
S. Sharan;Wenqing Zheng;Kuo-Feng Hsu;Jiarong Xing;Ang Chen;Zhangyang Wang
TCP拥塞控制（CC）的最新进展通过深度强化学习（RL）方法取得了巨大成功，该方法使用前馈神经网络（NN）来学习复杂的环境条件并做出更好的决策。然而，这种“黑盒”策略缺乏可解释性和可靠性，并且由于使用复杂的NN，它们通常需要在传统TCP数据路径之外操作。本文提出了一种新的两阶段解决方案，以实现两全其美：首先训练一个深度RL代理，然后将其（过）参数化NN策略提取为白盒，轻量级规则，以符号表达式的形式，更容易理解和在约束环境中实现。在我们的建议的核心是一个新的符号分支算法，使规则能够知道的上下文在各种网络条件，最终转换成一个符号树的NN政策。经过提炼的符号规则保留了最先进的NN策略，并经常提高性能，同时比标准神经网络更快，更简单。我们验证我们的蒸馏符号规则的性能在模拟和仿真环境。我们的代码可以在https://github.com/VITA-Group/SymbolicPCC上找到。
Recent advances in TCP congestion control (CC) have achieved tremendous success with deep reinforcement learning (RL) approaches, which use feedforward neural networks (NN) to learn complex environment conditions and make better decisions. However, such"black-box"policies lack interpretability and reliability, and often, they need to operate outside the traditional TCP datapath due to the use of complex NNs. This paper proposes a novel two-stage solution to achieve the best of both worlds: first to train a deep RL agent, then distill its (over-)parameterized NN policy into white-box, light-weight rules in the form of symbolic expressions that are much easier to understand and to implement in constrained environments. At the core of our proposal is a novel symbolic branching algorithm that enables the rule to be aware of the context in terms of various network conditions, eventually converting the NN policy into a symbolic tree. The distilled symbolic rules preserve and often improve performance over state-of-the-art NN policies while being faster and simpler than a standard neural network. We validate the performance of our distilled symbolic rules on both simulation and emulation environments. Our code is available at https://github.com/VITA-Group/SymbolicPCC.