Collaborative Research: CT-ISG: Accurate Sampling of the Internet for Effective Anomaly Detection

点击翻译按钮获取中文摘要

Title: Collaborative Research: CT-ISG: Accurate Sampling of the Internetfor Effective Anomaly DetectionAbstract:Sampled traffic data has been increasingly used as input for anomalydetection systems, as the high link speeds make it impossible toexamine each and every packet. This raises an important question ofwhether sampling has a (negative) impact on the accuracy/effectivenessof anomaly detection, and if so how to mitigate this effect.Intellectual Merit: This project systematically studies the questionmentioned above from the following three angles. First, we willidentify traffic features that are critical for a wide range ofanomaly detection schemes and quantify how much they are distorted byvarious sampling schemes. Second, we will design new sampling ormeasurement techniques that preserve enough accuracy to supporteffective anomaly detection, while being cost-effective andlight-weight. Third, we will study how to correlate the NetFlowsamples obtained at the edge routers with the information-rich datagenerated using existing data streaming algorithms, for much betteranomaly detection than pure sampling. The new scientific knowledgelearned through this research will provide us with much bettertechnologies to monitor large high-speed networks for anomalousbehaviors.Broader impact: The results will be broadly disseminated throughpublications, invited talks and tutorials, and open-sourcing ofsoftware developed for this project. The PIs' collaboration withtier-1 ISP's will facilitate the transfer of technology from researchenvironment to actual managing of production networks. Researchresults will be incorporated into information security curriculum.Both PIs have been actively engaging under-represented groups inresearch and higher education and will continue and expand theseefforts.