Inter-data-center network traffic prediction with elephant flows
Inter-data-center network traffic prediction with elephant flows
复制标题
利用大象流进行数据中心间网络流量预测
DOI：
10.1109/noms.2016.7502814
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
W. Xu
中科院分区：
文献类型：
--
作者：
Yi Li;Hong Liu;Wenjun Yang;Dianming Hu;W. Xu
随着大规模Internet应用的不断增加，数据中心之间的数据传输变得越来越普遍。传统的数据中心间传输存在利用率低和拥塞的问题，流量预测是优化数据中心间传输的重要方法。dc间流量比许多其他类型的网络流量更难预测，因为它主要由少数大型应用程序控制。我们提出了一个显著降低预测误差的模型。在该模型中，我们将小波变换与人工神经网络（ANN）相结合，以提高预测精度。具体来说，我们明确地在我们的预测模型中加入了大象流的信息，大象流是dc间网络中最不可预测但占主导地位的流量。为了减少象流信息的监控开销，我们添加了插值来填充象流中的未知值。我们证明，与现有方法相比，我们可以将预测误差降低5%~10%。我们的预测已经在b百度（中国最大的互联网公司之一）投入使用，有助于降低峰值网络带宽。
With the ever increasing number of large scale Internet applications, inter data center (inter-DC) data transfers are becoming more and more common. Traditional inter-DC transfers suffers from both low-utilization and congestion, and traffic prediction is an important method to optimize these transfers. Inter-DC traffic is harder to predict than many other types of network traffic, because it is dominated by a few large applications. We propose a model that significantly reduces the prediction errors. In our model, we combine wavelet transform with artificial neural network (ANN) to improve prediction accuracy. Specifically, we explicitly add information of elephant flows, the least predictable yet dominating traffic in inter-DC network, into our prediction model. To reduce the amount of monitoring overhead for the elephant flow information, we added interpolation to fill in the unknown values in the elephant flows. We demonstrate that we can reduce prediction errors over existing methods by 5%~10%. Our prediction is already in production at Baidu, one of the largest Internet companies in China, helping reducing the peak network bandwidth.