Searching for Anomalies Over Composite Hypotheses
Searching for Anomalies Over Composite Hypotheses
复制标题
DOI：
10.1109/tsp.2020.2971438
复制
发表时间：
2020-01-01
影响因子：
5.4
通讯作者：
Zhao, Qing
中科院分区：
文献类型：
--
作者：
Hemo, Bar;Gafni, Tomer;Zhao, Qing
考虑了检测多个进程中的异常的问题。我们考虑一个复合假设情况，其中观察过程时得出的测量结果遵循具有未知参数（向量）的共同分布，其值位于正常或异常参数空间中，具体取决于其状态。目标是一种顺序搜索策略，可最大限度地减少受错误概率约束的预期检测时间。我们开发了一种具有以下所需属性的确定性搜索算法。首先，当不知道有关过程状态的附加信息时，当错误概率接近零时，所提出的算法在最小化检测延迟方面是渐近最优的。其次，当零假设下的参数值已知并且对于所有正常过程都相等时，所提出的算法也是渐近最优的，更好的检测时间由真正的零状态决定。第三，当原假设下的参数值未知，但已知所有正常过程都相等时，所提出的算法在实现随着检测延迟衰减到零的错误概率方面是一致的。最后，为有限样本状态建立了所提出算法下的错误概率的明确上限。在合成数据集和 DARPA 入侵检测数据集上进行了广泛的实验，证明了所提出的算法比现有方法具有更强的性能。
The problem of detecting anomalies in multiple processes is considered. We consider a composite hypothesis case, in which the measurements drawn when observing a process follow a common distribution with an unknown parameter (vector), whose value lies in normal or abnormal parameter spaces, depending on its state. The objective is a sequential search strategy that minimizes the expected detection time subject to an error probability constraint. We develop a deterministic search algorithm with the following desired properties. First, when no additional side information on the process states is known, the proposed algorithm is asymptotically optimal in terms of minimizing the detection delay as the error probability approaches zero. Second, when the parameter value under the null hypothesis is known and equal for all normal processes, the proposed algorithm is asymptotically optimal as well, with better detection time determined by the true null state. Third, when the parameter value under the null hypothesis is unknown, but is known to be equal for all normal processes, the proposed algorithm is consistent in terms of achieving error probability that decays to zero with the detection delay. Finally, an explicit upper bound on the error probability under the proposed algorithm is established for the finite sample regime. Extensive experiments on synthetic dataset and DARPA intrusion detection dataset are conducted, demonstrating strong performance of the proposed algorithm over existing methods.