ATD: Scanning Dynamic Spatial-Temporal Discrete Events for Threat Detection

这个项目的主要研究目标是开发一个从时空离散事件数据中检测异常的统计框架。如今，分布在空间和时间上的大量此类事件数据正变得越来越多地可用于各种应用，例如人类活动数据、社交网络数据和犯罪数据。离散事件的观测可以在连续的时间和地点发生，并且可以有对这些事件的复杂文本描述。离散事件数据包含丰富的相关性和因果信息，这些信息可能被用来推断底层系统的动态并检测威胁。该项目旨在开发统计方法，利用离散事件在威胁检测中挖掘这一潜力，并解决算法和计算方面的挑战。所开发的方法将超越现状模型估计，考虑更一般的统计推断问题，如假设检验和基于似然的推断。该方法具有较强的通用性，适用于各种离散事件数据。该项目将在亚特兰大警察局收集的大规模犯罪数据集上具体展示所开发方法的有效性。近年来，点过程模型已被证明是一种捕捉离散事件相关性结构的有效模型。虽然在估计自激发的时空点过程模型方面已经取得了很大的成功，但如何利用这些模型进行异常检测仍然不清楚，因为(1)检测(可以被视为假设检验)本质上不同于估计，它涉及不同类型的统计和性能度量；(2)在各种情况下，在广泛的空间区域内存在大量离散事件，目标是检测相关事件的一小群，这相当于大海捞针，因此需要开发强大的和计算高效的统计；(3)正常或参考状态可能是复杂的和动态的，方法需要适应缓慢时变的正常状态。该项目将解决这些挑战，并为两个相关的基本问题提供答案：如何使用点过程模型从大量数据中检测相关事件的集群，以及如何估计时变背景正常模式。这项工作将促进扫描统计研究的发展，并在伪似然估计和强化学习之间建立一种新的联系。所开发的方法将在犯罪数据分析的具体应用中进行测试。拟议的教育活动将让所有级别的学生接受严格的数学和统计培训，并获得实际操作的数据分析技能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The overarching research objective of this project is to develop a statistical framework for detecting anomalies from spatial-temporal discrete event data. Nowadays, a large volume of such event data dispersed over space and time are becoming increasingly available in a wide variety of applications, such as human activity data, social network data, and crime data. The observations of the discrete events can occur in continuous time and locations, and there can be a complex text description of such events. The discrete event data contain rich correlation and causal information, which can potentially be used to infer the dynamics of the underlying systems and detect threats. The project aims to develop statistical methods to harvest this potential in threat detection using discrete events and address the algorithmic and computational challenges. The developed methods will go beyond the status-quo model estimation by considering more general statistical inference problems such as hypothesis tests and likelihood-based inference. The developed methods are general and can be used for various discrete event data. The project will specifically demonstrate the effectiveness of the developed methods on a large-scale crime dataset collected by the Atlanta Police Department. Recently, point process models have been proven an effective model for capturing the correlation structure in discrete events. While much success has been achieved in estimating the self-exciting spatial-temporal point process models, it remains unclear how one can perform anomaly detection leveraging these models, since (1) detection (which can be cast as hypothesis test) is inherently different from estimation, which involves different kinds of statistics and performance metrics; (2) in various situations, there is a large number of discrete events over broad spatial areas, and the goal is to detect a small cluster of related events, which amounts to "finding a needle in a haystack", thus there is a need to develop powerful and computationally efficient statistics; (3) the normal or reference state can be complex and dynamic and methods need to adapt to the slowly time-varying normal state. The project will address these challenges and provide answers to two related fundamental questions: how to detect clusters of correlated events from a large amount of data using the point process model, and how to estimate time-varying background normal pattern. The proposed work will advance the state-of-art for scan statistic research and build a novel connection between pseudo-likelihood estimation and reinforcement learning. The developed methods will be tested in a specific application of crime data analysis. The proposed education activities will involve students at all levels in rigorous mathematical and statistical training and gain hands-on data analysis skills.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.