Change detection in enterprise-wide computer network traffic

企业范围内计算机网络流量的变化检测已被业界和情报部门认可，数据科学技术具有提供下一代网络安全防御的潜力。在典型的企业计算机网络中，有大量可用的大容量数据源，这些数据源可以发现和预防网络攻击和其他恶意网络活动。虽然传统的网络防御系统侧重于检测数据包中的强签名，例如内容感知防火墙和防病毒软件，但使用更多基于统计、概率模型的技术来识别更微妙的入侵尝试，在很大程度上是一个未被充分利用的机会。这种方法的潜在优势是能够从历史数据中学习计算机和网络行为的正常模式，这样就可以发现异常情况，否则就不会突出；一个例子是使用合法凭证进行不寻常的网络遍历。这里的兴趣将集中于检测大规模汇总统计的概率分布中的重大时间变化，例如不同服务端口的相对流行程度，企业内计算机连接的国家，或者可能的网络的其他一些本地特征。首选的方法将是基于贝叶斯模型的变点分析，这需要计算模拟技术，如马尔可夫链蒙特卡罗。开发的任何方法都需要可扩展，以便在整个网络中进行实际部署。使用大数据平台进行一些探索性数据分析是必要的，以确定数据中的主要结构，指导模型构建过程。该项目与EPSRC全球不确定性和数字经济战略主题以及统计和应用概率研究主题保持一致。

Change detection in enterprise-wide computer network trafficIt is recognised by industry and the intelligence services that data science techniques have the potential to provide the next generation of cyber-security defences. Inside a typical enterprise computer network, a number of high-volume data sources are available which can enable the discovery and prevention of cyber-attacks and other nefarious network activity. Whilst traditional systems of cyber-defence focus on detecting strong signatures in packets, such as content-aware firewalls and antivirus software, there is a largely under-exploited opportunity to use more statistical, probabilistic model-based techniques for identifying more subtle intrusion attempts. The potential advantage of such approaches is the ability to learn, from historical data, normal patterns of computer and network behaviour, so that anomalies can then be detected which would not stand out otherwise; one example is unusual network traversal using legitimate credentials.Interest here will focus on detecting significant temporal changes in the probability distribution of large-scale summary statistics, such as the relative popularity of different service ports, the countries connected to by computers within an enterprise, or possibly some other local characteristic of the network. The preferred approach will be Bayesian model-based changepoint analysis, which requires computational simulation techniques such as Markov chain Monte Carlo. Any methods developed will need to be scalable for realistic deployment across an entire network. Some exploratory data analysis using big data platforms will be necessary for identifying the main structures in the data, to guide the model-building process.The project is aligned to the EPSRC Global Uncertainty and Digital Economy strategic themes, and the Statistics and Applied Probability research theme.