ATD: Efficient online detection based on multiple sensors, with applications to cybersecurity and discovery of biological threats

本项目重点研究基于同时观测多个数据序列的威胁检测方案。正确使用多个信息源或多个传感器，可确保检测过程的高灵敏度。然而，多种不同的信息流可能导致假警报。结合混合类型的数据以产生快速且无错误的检测的最佳统计技术是什么？提出的统计方法通过检测多维数据分布中的异质性和异常以及定位变化点来处理潜在的威胁。假设多个传感器同时收集和报告数据序列。当重大事件发生和潜在威胁出现时，一个或几个序列的分布会发生变化。目标是在低误报率的情况下，尽快发现威胁。多序列的最优威胁检测算法的推导将基于最近发展的序列实验中的多重比较理论和方法。PI和他的学生引入的这些新技术导致了多个假设的测试，这些假设可以在低预期采样成本下控制家庭误差率和家庭功率。提出了几种快速多传感器变化点检测方法。基于新方法，将开发CUSUM，贝叶斯和渐近点最优变点检测工具的类似物，以控制假警报的概率，错过的发现率，并在这些约束下最小化平均检测延迟。通过发现分布、模式和趋势的变化来快速检测威胁是质量控制、市场分析、流行病学、气候学、目标跟踪和其他领域最重要的问题之一。在广泛的应用领域中，该项目特别侧重于检测网络安全和生物威胁，如流行病和生物恐怖袭击。该项目将提供通用工具，以便在实际情况下迅速对威胁性异常作出反应，例如：(i)根据不同区域的地理空间公共卫生监测数据识别流行病前的模式并发出流行病威胁的信号；（ii）根据多个数据流检测计算机威胁和网络安全漏洞；（iii）从通信网络的实际分析中检测潜在威胁。变化点检测在多数据流上的重要现代应用出现在DNA测序中。在多个数据流的顺序变化点分析中利用先验信息的可能性为广泛的应用打开了大门。它将能够预测威胁，并对表现出两阶段或多阶段行为的一些过程进行预测，例如流行病和流行病间期、经济增长和衰退以及能源价格飙升。开发的快速变化点检测方法将用于早期发现未知目标和入侵、欺诈活动、重要地点的异常行为、发现和分类流行病前趋势，以及预防流行病和恐怖袭击。

The project focuses on threat detection schemes based on simultaneously observed multiple data sequences. The proper use of multiple sources of information, or multiple sensors, ensures high sensitivity of the detection procedure. However, multiple streams of diverse information can cause false alarms. What are the optimal statistical techniques of combining mixed types of data to yield a quick and error-free detection? Proposed statistical methods deal with potential threats by detecting heterogeneities and anomalies and locating change points in the distribution of multidimensional data. It is assumed that a number of sensors simultaneously collect and report data sequences. When a significant event occurs and a potential threat appears, the distribution of one or several sequences changes. The goal is to detect a threat as soon as possible, subject to a low rate of false alarms. Derivation of optimal threat detection algorithms on multiple sequences will be based on the recently developed theory and methodology of multiple comparisons in sequential experiments. These new techniques introduced by the PI and his student led to tests of multiple hypotheses that control both the familywise error rate and the familywise power at a low expected sampling cost. This suggests several approaches to the quick multi-sensor change-point detection. Analogues of CUSUM, Bayesian, and asymptotically pointwise optimal change-point detection tools will be developed based on the new methodology in order to control the probability of a false alarm, the missed discovery rate, and to minimize the mean detection delay under these constraints. Quick detection of threats by discovering changes in distributions, patterns, and trends is one of the most vital problems in quality control, market analysis, epidemiology, climatology, target tracking, and other fields. Among wide areas of application, this project particularly focuses on detecting breaches in cyber security and biological threats such as epidemics and bioterrorist attacks. The project will provide general tools for the prompt reaction to threatening anomalies in real situations such as (i) recognizing a pre-epidemic pattern and signalling an epidemic threat based on geospatial public health surveillance data in different regions, (ii) detecting computer threats and breaches in cyber security, based on multiple data streams, and (iii) detecting potential threats from extual analysis of communication networks. An important modern application of change-point detection on multiple data streams appears in DNA sequencing. The possibility of utilizing the prior information in sequential change-point analysis of multiple data streams opens doors for wide applications. It will allow to predict threats and make forecasts for a number of processes that exhibit a two-phase or multi-phase behavior, such as the epidemics and inter-epidemic periods, economic growth and recession, and spikes in energy prices. Developed methods of fast change-point detection will be used for the early detection of unknown targets and intrusions, fraud activity, unusual behavior at vital locations, detection and classification of pre-epidemic trends, and also, for the prevention of epidemics and terrorist attacks.