Anomaly-Tolerant Network Traffic Estimation via Noise-Immune Temporal Matrix Completion Model
Anomaly-Tolerant Network Traffic Estimation via Noise-Immune Temporal Matrix Completion Model
复制标题
通过抗噪声时间矩阵完成模型进行异常容忍网络流量估计
DOI：
10.1109/jsac.2019.2904347
复制
发表时间：
2019-03
影响因子：
16.4
通讯作者：
Wang Ruchuan
中科院分区：
文献类型：
--
作者：
Xiao Fu;Chen Lei;Zhu Hai;Hong Richang;Wang Ruchuan
准确估计起始目的地（OD）网络流量对于网络管理和容量规划至关重要。然而，潜在的网络异常和复杂的噪声使得这一目标难以实现。现有的网络流量估计方法通常独立于异常检测来估算网络流量，这忽略了两个任务之间的潜在关系以帮助彼此获得更好的性能。此外，这些方法只能适用于简单的高斯或异常噪声假设，无法应用于实际应用中更复杂的噪声分布。为了解决这些问题，我们提出了一种新颖的异常容忍网络流量估计方法，用于同时估计网络流量和检测网络异常。具体来说，利用流量矩阵固有的低秩属性和时间特性，我们将网络流量估计问题表述为噪声免疫时间矩阵完成（NiTMC）模型，其中复杂噪声通过混合高斯（MoG）进行拟合，网络异常通过$L_{2,1}$范数正则化进行平滑。此外，我们还设计了一种基于期望最大化（EM）和块坐标更新（BCU）方法的保证收敛的优化算法来求解所提出的模型。此外，为了处理大规模网络问题，我们通过采用随机近端梯度下降（SPGD）方法开发了一种可扩展且内存高效的算法。最后，在真实数据集上进行的大量实验表明，我们提出的 NiTMC 模型优于之前广泛使用的网络流量估计方法。
Accurately estimating origin-destination (OD) network traffic is crucial for network management and capacity planning. However, the potential network anomaly and complex noise make this goal difficult to achieve. Existing network traffic estimation methods usually impute network traffic independent of anomaly detection, which ignores the potential relationship between the two tasks to help each other in achieving better performance. Moreover, these approaches can only be suitable for simple Gaussian or outlier noise assumptions, which cannot be applied to more complex noise distributions in practical applications. To address these issues, we propose a novel anomaly-tolerant network traffic estimation approach for simultaneously estimating network traffic and detecting network anomaly. Specifically, by utilizing the inherent low-rank property and temporal characteristic of traffic matrix, we formulate the network traffic estimation problem as a noise-immune temporal matrix completion (NiTMC) model, where the complex noise is fitted by mixture of Gaussian (MoG), and the network anomaly is smoothed by the $L_{2,1}$ -norm regularization. In addition, we also design a convergence-guaranteed optimization algorithm based on the expectation maximization (EM) and block coordinate update (BCU) methods to solve the proposed model. Furthermore, to deal with large-scale network problems, we develop a scalable and memory-efficient algorithm by employing stochastic proximal gradient descent (SPGD) method. Finally, the extensive experiments performed on real datasets demonstrate that our proposed NiTMC model outperforms the previously widely used network traffic estimation methods.