ATD: Collaborative Research: Extremal Dependence and Change-Point Detection Methods for High-Dimensional Data Streams with Applications to Network Cybersecurity
ATD:协作研究:高维数据流的极端依赖性和变点检测方法及其在网络网络安全中的应用
基本信息
- 批准号:1830175
- 负责人:
- 金额:$ 8.8万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2018
- 资助国家:美国
- 起止时间:2018-08-01 至 2021-07-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
The project is motivated by the need to develop advanced network monitoring tools coupled with automated statistical methods for the quick detection of Internet traffic anomalies due to ongoing attacks or impending cybersecurity threats. Emphasis is placed on detecting cybersecurity threats such as highly distributed malware infections, which can launch coordinated and crippling distributed denial of service attacks on the nation's Internet infrastructure. This will be achieved through a study of the so-called darknet traffic data. Malicious actors in the network systematically probe the Internet space for vulnerable or misconfigured devices. In doing so, they automatically send data to the entire Internet address space, which includes the space of unused Internet addresses. This destined-to-nowhere traffic is indicative of malware infection attempts or stealthy vulnerability scanning. The investigators aim to develop and deploy specialized tools that allow cyber-security analysts to efficiently analyze darknet traffic data. The research involves a team of computer engineers and statisticians, who will work closely together to implement a prototype system for detecting as well as mapping and identifying world-wide malicious activity in the Internet. The project will create and communicate to the public a set of simple-to-interpret risk indices that summarize the current darknet threat activity. This effort will potentially enable the prevention and mitigation of cybersecurity network traffic threats.Understanding Internet threats, which continue to evolve due to the dynamic nature of Internet actors and the rapid expansion of the Internet of Things ecosystem, requires adequate data at fine-grained spatial and temporal scales. The project team has access to unique cyber-security data collected at Merit Network, Inc. that capture Internet-wide activity including network scanning, malware propagation, denial of service attacks, and network outages. This data consists of unsolicited Internet traffic destined to a routed but unused Internet address space, referred to as a darknet. This project will develop algorithmic and software infrastructure to collect and organize darknet data into high-dimensional, multivariate data streams, and will study statistical methods based on (i) extremal dependence, (ii) change-point detection, and/or (iii) high-dimensional sparse signal detection and recovery to inform the construction of Internet threat indices that quantify the risk of malicious scanning, degree of network vulnerability, risk of denial of service attacks, etc. Statistics of extremes in high-dimensional setting is a challenging problem since it requires the modeling/estimation of an infinite-dimensional parameter---the spectral measure. Using multivariate regular variation, this project will study novel hyper-graphical models that quantify and provide interpretable abstractions for the simultaneous occurrence of extremes in high-dimensions. Using limit theory for maxima of dependent variables, the project team will address open theoretical problems on the characterization of extremal dependence hyper-graphs and sparse signal detection in high-dimension. This analysis will lead to the development of novel threat indices that exhibit spatial dependence that will be analyzed with fast, scalable change-point detection algorithms. The new change-point methodology is designed to achieve large computational gains vis-a-vis standard approaches without compromising statistical accuracy and would be a significant contribution to the analysis of large data streams.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
该项目的动机是需要开发先进的网络监控工具,并结合自动统计方法,以快速检测由于持续攻击或即将到来的网络安全威胁而导致的互联网流量异常。重点放在检测网络安全威胁,如高度分布式的恶意软件感染,它可以对国家的互联网基础设施发起协调和破坏性的分布式拒绝服务攻击。 这将通过研究所谓的暗网流量数据来实现。网络中的恶意行为者系统地探测互联网空间,寻找易受攻击或配置错误的设备。在这样做时,它们会自动将数据发送到整个互联网地址空间,其中包括未使用的互联网地址空间。这种无处可去的流量表示恶意软件感染尝试或隐形漏洞扫描。 调查人员的目标是开发和部署专门的工具,使网络安全分析师能够有效地分析暗网流量数据。这项研究涉及一组计算机工程师和统计学家,他们将密切合作,实施一个原型系统,用于检测、绘制和识别互联网上的世界范围恶意活动。该项目将创建并向公众传达一组简单易懂的风险指数,总结当前的暗网威胁活动。 由于互联网参与者的动态性和物联网生态系统的快速扩张,互联网威胁不断演变,了解互联网威胁需要在细粒度的空间和时间尺度上获得足够的数据。项目团队可以访问Merit Network,Inc.收集的唯一网络安全数据。捕获Internet范围内的活动,包括网络扫描、恶意软件传播、拒绝服务攻击和网络中断。这些数据由未经请求的互联网流量组成,目的地是路由但未使用的互联网地址空间,称为暗网。该项目将开发算法和软件基础设施,以收集和组织暗网数据到高维,多变量数据流,并将研究基于(i)极值依赖,(ii)变点检测和/或(iii)高维稀疏信号检测和恢复的统计方法,以告知构建互联网威胁指数,量化恶意扫描的风险,网络脆弱性的程度,拒绝服务攻击的风险等。高维环境中的极端情况的统计是一个具有挑战性的问题,因为它需要对无限维参数--谱度量进行建模/估计。使用多元规则变化,这个项目将研究新的超图形模型,量化和提供可解释的抽象的同时发生的极端在高维度。利用因变量最大值的极限理论,项目团队将解决关于极值依赖超图的表征和高维稀疏信号检测的开放性理论问题。这种分析将导致开发新的威胁指数,表现出空间依赖性,将与快速,可扩展的变化点检测算法进行分析。新的变点方法的目的是在不影响统计准确性的情况下,实现与标准方法相比的大计算增益,并将对大型数据流的分析做出重大贡献。该奖项反映了NSF的法定使命,并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。
项目成果
期刊论文数量(9)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Broader impacts of network monitoring: Its role in government, industry, technology, and beyond
- DOI:10.1080/08982112.2021.1974036
- 发表时间:2021-10
- 期刊:
- 影响因子:2
- 作者:Nathaniel T. Stevens;James D. Wilson;Anne R. Driscoll;I. Mcculloh;G. Michailidis;Cécile Paris;K. Paynabar;M. Perry;Mostafa Reisi-Gahrooei;Srijan Sengupta;Ross Sparks
- 通讯作者:Nathaniel T. Stevens;James D. Wilson;Anne R. Driscoll;I. Mcculloh;G. Michailidis;Cécile Paris;K. Paynabar;M. Perry;Mostafa Reisi-Gahrooei;Srijan Sengupta;Ross Sparks
Change Point Estimation in a Dynamic Stochastic Block Model
- DOI:
- 发表时间:2018-12
- 期刊:
- 影响因子:0
- 作者:M. Bhattacharjee;M. Banerjee;G. Michailidis
- 通讯作者:M. Bhattacharjee;M. Banerjee;G. Michailidis
The interdisciplinary nature of network monitoring: Advantages and disadvantages
网络监控的跨学科性质:优点和缺点
- DOI:10.1080/08982112.2021.1974034
- 发表时间:2021
- 期刊:
- 影响因子:2
- 作者:Stevens, Nathaniel T.;Wilson, James D.;Driscoll, Anne R.;McCulloh, Ian;Michailidis, George;Paris, Cecile;Parker, Peter;Paynabar, Kamran;Perry, Marcus B.;Reisi-Gahrooei, Mostafa
- 通讯作者:Reisi-Gahrooei, Mostafa
Foundations of network monitoring: Definitions and applications
- DOI:10.1080/08982112.2021.1974033
- 发表时间:2021-10
- 期刊:
- 影响因子:2
- 作者:Nathaniel T. Stevens;James D. Wilson;Anne R. Driscoll;I. Mcculloh;G. Michailidis;Cécile Paris;K. Paynabar;M. Perry;Mostafa Reisi-Gahrooei;Srijan Sengupta;Ross Sparks
- 通讯作者:Nathaniel T. Stevens;James D. Wilson;Anne R. Driscoll;I. Mcculloh;G. Michailidis;Cécile Paris;K. Paynabar;M. Perry;Mostafa Reisi-Gahrooei;Srijan Sengupta;Ross Sparks
Fast and Scalable Algorithm for Detection of Structural Breaks in Big VAR Models
- DOI:10.1080/10618600.2021.1950005
- 发表时间:2021-08
- 期刊:
- 影响因子:2.4
- 作者:Abolfazl Safikhani;Yue Bai;G. Michailidis
- 通讯作者:Abolfazl Safikhani;Yue Bai;G. Michailidis
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
George Michailidis其他文献
Asymptotics for <math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si4.gif" display="inline" overflow="scroll" class="math"><mi>p</mi></math>-value based threshold estimation under repeated measurements
- DOI:
10.1016/j.jspi.2016.01.009 - 发表时间:
2016-07-01 - 期刊:
- 影响因子:
- 作者:
Atul Mallik;Bodhisattva Sen;Moulinath Banerjee;George Michailidis - 通讯作者:
George Michailidis
Queueing Networks of Random Link Topology: Stationary Dynamics of Maximal Throughput Schedules
- DOI:
10.1007/s11134-005-0858-x - 发表时间:
2005-05-01 - 期刊:
- 影响因子:0.700
- 作者:
Nicholas Bambos;George Michailidis - 通讯作者:
George Michailidis
DNEA: an R package for fast and versatile data-driven network analysis of metabolomics data
- DOI:
10.1186/s12859-024-05994-1 - 发表时间:
2024-12-18 - 期刊:
- 影响因子:3.300
- 作者:
Christopher Patsalis;Gayatri Iyer;Marci Brandenburg;Alla Karnovsky;George Michailidis - 通讯作者:
George Michailidis
Statistica Sinica Preprint No: SS-2022-0323
《统计》预印本编号:SS-2022-0323
- DOI:
- 发表时间:
- 期刊:
- 影响因子:0
- 作者:
Abhishek Kaul;George Michailidis;Statistica Sinica - 通讯作者:
Statistica Sinica
Preface: Computational biomedicine
- DOI:
10.1007/s10479-018-3116-4 - 发表时间:
2019-01-14 - 期刊:
- 影响因子:4.500
- 作者:
Anton Kocheturov;Panos Pardalos;George Michailidis - 通讯作者:
George Michailidis
George Michailidis的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('George Michailidis', 18)}}的其他基金
ATD: Spatio-Temporal Modeling for Identifying Changes in Land Use
ATD:识别土地利用变化的时空模型
- 批准号:
2334735 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Change Point Detection for Data with Network Structure
网络结构数据变点检测
- 批准号:
2348640 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: Geospatial Modeling and Risk Mitigation for Human Movement Dynamics under Hurricane Threats
合作研究:ATD:飓风威胁下人类运动动力学的地理空间建模和风险缓解
- 批准号:
2319552 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: IMR: MM-1A: Scalable Statistical Methodology for Performance Monitoring, Anomaly Identification, and Mapping Network Accessibility from Active Measurements
合作研究:IMR:MM-1A:用于性能监控、异常识别和主动测量映射网络可访问性的可扩展统计方法
- 批准号:
2319593 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Change Point Detection for Data with Network Structure
网络结构数据变点检测
- 批准号:
2210358 - 财政年份:2022
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
ATD: Spatio-Temporal Modeling for Identifying Changes in Land Use
ATD:识别土地利用变化的时空模型
- 批准号:
2124507 - 财政年份:2021
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
CDS&E: Statistical Methodology for Analysis and Forecasting with Large Scale Temporal Data
CDS
- 批准号:
1821220 - 财政年份:2018
- 资助金额:
$ 8.8万 - 项目类别:
Continuing Grant
BIGDATA: Collaborative Research: IA: F: Too Interconnected to Fail? Network Analytics on Complex Economic Data Streams for Monitoring Financial Stability
BIGDATA:协作研究:IA:F:互联性太强以至于不会失败?
- 批准号:
1632730 - 财政年份:2016
- 资助金额:
$ 8.8万 - 项目类别:
Continuing Grant
CyberSEES: Type 2: Collaborative Research: Tenable Power Distribution Networks
CyberSEES:类型 2:协作研究:可维持的配电网络
- 批准号:
1540093 - 财政年份:2015
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: Statistical Methodology for Network based Integrative Analysis of Omics Data
合作研究:基于网络的组学数据综合分析统计方法
- 批准号:
1545277 - 财政年份:2015
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
相似海外基金
Collaborative Research: ATD: Fast Algorithms and Novel Continuous-depth Graph Neural Networks for Threat Detection
合作研究:ATD:用于威胁检测的快速算法和新颖的连续深度图神经网络
- 批准号:
2219956 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: a-DMIT: a novel Distributed, MultI-channel, Topology-aware online monitoring framework of massive spatiotemporal data
合作研究:ATD:a-DMIT:一种新颖的分布式、多通道、拓扑感知的海量时空数据在线监测框架
- 批准号:
2220495 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: Rapid Structure Recovery and Outlier Detection in Multidimensional Data
合作研究:ATD:多维数据中的快速结构恢复和异常值检测
- 批准号:
2319370 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: Geospatial Modeling and Risk Mitigation for Human Movement Dynamics under Hurricane Threats
合作研究:ATD:飓风威胁下人类运动动力学的地理空间建模和风险缓解
- 批准号:
2319552 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: Fast Algorithms and Novel Continuous-depth Graph Neural Networks for Threat Detection
合作研究:ATD:用于威胁检测的快速算法和新颖的连续深度图神经网络
- 批准号:
2219904 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: Rapid Structure Recovery and Outlier Detection in Multidimensional Data
合作研究:ATD:多维数据中的快速结构恢复和异常值检测
- 批准号:
2319371 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: Rapid Structure Recovery and Outlier Detection in Multidimensional Data
合作研究:ATD:多维数据中的快速结构恢复和异常值检测
- 批准号:
2319372 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
Collaborative Research: ATD: Geospatial Modeling and Risk Mitigation for Human Movement Dynamics under Hurricane Threats
合作研究:ATD:飓风威胁下人类运动动力学的地理空间建模和风险缓解
- 批准号:
2319551 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
ATD: Collaborative Research: A Geostatistical Framework for Spatiotemporal Extremes
ATD:协作研究:时空极值的地统计框架
- 批准号:
2220523 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant
ATD: Collaborative Research: A Geostatistical Framework for Spatiotemporal Extremes
ATD:协作研究:时空极值的地统计框架
- 批准号:
2220529 - 财政年份:2023
- 资助金额:
$ 8.8万 - 项目类别:
Standard Grant