Hypersparse Neural Network Analysis of Large-Scale Internet Traffic
Hypersparse Neural Network Analysis of Large-Scale Internet Traffic
复制标题
DOI：
10.1109/hpec.2019.8916263
复制
发表时间：
2019-04
期刊：
影响因子：
--
通讯作者：
J. Kepner;Kenjiro Cho;K. Claffy;V. Gadepally;P. Michaleas;Lauren Milechin
中科院分区：
文献类型：
--
作者：
J. Kepner;Kenjiro Cho;K. Claffy;V. Gadepally;P. Michaleas;Lauren Milechin
互联网正在改变我们的社会，需要对互联网流量进行定量了解。我们的团队收集和管理最大的公开互联网流量数据，包含500亿个数据包。利用一种新的hypersarse神经网络分析的“视频”流的流量使用10,000个处理器在麻省理工学院超级云揭示了一个新的现象：否则看不见的叶节点和孤立的链接在互联网流量的重要性。我们的神经网络方法进一步表明，一个双参数修改的Zipf-Mandelbrot分布准确地描述了各种各样的源/目的地统计数据的移动样本窗口范围从100，000到100，000，000包跨越几年和大陆的集合。推断出的模型参数区分不同的网络流，并且模型叶参数与不同底层网络拓扑中的流量的分数强烈相关。超解析神经网络管道具有高度适应性，不同的网络统计和训练模型可以通过对图像滤波器函数的简单更改来合并。
The Internet is transforming our society, necessitating a quantitative understanding of Internet traffic. Our team collects and curates the largest publicly available Internet traffic data containing 50 billion packets. Utilizing a novel hypersparse neural network analysis of “video” streams of this traffic using 10,000 processors in the MIT SuperCloud reveals a new phenomena: the importance of otherwise unseen leaf nodes and isolated links in Internet traffic. Our neural network approach further shows that a two-parameter modified Zipf-Mandelbrot distribution accurately describes a wide variety of source/destination statistics on moving sample windows ranging from 100,000 to 100,000,000 packets over collections that span years and continents. The inferred model parameters distinguish different network streams and the model leaf parameter strongly correlates with the fraction of the traffic in different underlying network topologies. The hypersparse neural network pipeline is highly adaptable and different network statistics and training models can be incorporated with simple changes to the image filter functions.