ATD: Anomaly Detection with Confidence and Precision

检测威胁对国家和全球安全至关重要。由于目前有大量的监视和其他类型的数据，因此需要一种系统和正式的定量方法来检测威胁。威胁通常出现在异常行为之前;因此，早期威胁检测成为异常行为检测。从统计学上讲，异常行为的检测本质上是从“通常”分布或“通常”关系中检测离群值。虽然错过威胁可能会对社会产生破坏性影响，但错误检测也会产生负面影响。该项目旨在开发一个新的异常值检测框架，在该框架下，检测的置信水平根据误报水平制定，并精确确定，以帮助决策者进行更知情的资源规划。这一发展为利用广泛的数据源进行威胁检测提供了新的统计方法。这种异常值检测方法和置信水平确定工具是一般的统计方法，形成了许多威胁检测和风险评估问题的有用框架。它们将丰富统计学的理论和方法，产生一个新的统计分析工具包，并有助于一般数据科学，因为离群值识别是有效的下游分析数据清理的关键阶段。调查人员将积极从事与研究生和本科生的教育和研究培训相关的活动，特别是吸引少数民族和女性学生进入统计学和统计应用领域，并向他们介绍对全球和国家安全具有重要意义的领域，尽管离群值检测算法已经得到广泛研究大多数现有方法不提供不确定性评估，并且依赖于特定规则来进行判断。本项目正在开发的“一致性离群值检测”（COD）框架可以克服这一缺点，并提供理论上有保证的置信水平的检测。这一开发基于机器学习和统计学中最先进的非参数预测推理工具，称为共形预测。 它可以提供准确的风险和不确定性的评估，对数据的假设很少，可以广泛应用。在这个新的COD框架下，该项目探索了两个离群值检测程序。第一种是无分布的，适用于任何提供受试者之间成对相似性度量的数据集。它可以用于在反恐监视中经常遇到的广泛类别的非常规数据集的离群值检测（例如，具有单词使用频率相似性度量的文本数据，通信模式变化，语音相似性，网络变化等）。第二种是基于模型的程序，用于检测与“通常”关系或行为的异常偏差。该检测方法在一定的设置下对模型误指定具有鲁棒性。此外，由于异质性在大型数据集中很常见，因此该项目包括在一般个性化学习框架下将COD程序扩展到精确的上下文离群值检测。最后，该项目的目的是在一个特定的环境中展示的方法，很少观察到的空间和时间的计数过程中经常遇到的监测偏远地区。该奖项反映了NSF的法定使命，并被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Detection of threats is critical to national and global security. With tremendous amounts of surveillance and other types of data currently available, a systematic and formal quantitative approach to threat detection is needed. Threats are often preceded by abnormal behavior; early threat detection thus becomes detection of abnormal behavior. Statistically, detection of abnormal behavior is essentially detection of outliers from a "usual" distribution or a "usual" relationship. While missing a threat may have devastating impacts to society, a false detection also has negative impacts. This project aims to develop a new outlier detection framework under which the confidence level of detection is formulated in terms of the level of false positives and is precisely determined to aid decision makers for more informed resource planning. The development provides novel statistical approaches for threat detection utilizing a wide range of data sources. This outlier detection method and the confidence level determination tool are general statistical methods that form a useful framework for many threat detection and risk assessment problems. They will enrich the theory and methodology of statistics, produce a new statistical analytical toolkit, and contribute to general data science, since outlier identification is a crucial stage of data cleaning for valid downstream analysis. The investigators will actively engage in activities related to education and research training of graduate and undergraduate students, especially attracting minority and women students into the fields of statistics and statistical applications, and introducing them to areas that are important to global and national security.Although outlier detection algorithms have been extensively studied, most existing methods do not provide an uncertainty assessment and rely on an ad hoc rule to make judgment calls. The “Conformity Outlier Detection” (COD) framework under development in this project can overcome the shortcoming and provide detection with a theoretically guaranteed confidence level. This development is based on a state-of-art non-parametric predictive inference tool in machine learning and statistics, known as conformal prediction. It can provide accurate assessment of risk and uncertainty with little assumption on the data and can be applied broadly. Under this new COD framework, the project explores two outlier detection procedures. The first is distribution-free and is suitable for any data set that provides pairwise similarity measures between subjects. It can be used for outlier detection of a broad class of unconventional data sets often encountered in counter-terrorism surveillance (e.g. text data with word-use frequency similarity measure, communication pattern changes, voice similarities, network changes, and many others). The second is a model-based procedure to detect an abnormal deviation from a "usual" relationship or behavior. The detection method is robust against model misspecification under certain settings. In addition, since heterogeneity is commonly seen in large data sets, the project includes extension of the COD procedures to precision contextual outlier detection under the general individualized learning framework. Lastly, the project aims to demonstrate the approaches in a specific setting with sparely observed spatial and temporal count processes that are commonly encountered in surveillance of remote areas. This project will support one graduate student per year.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.