Low Latency Anomaly Detections with Imperfect Data Models

异常检测具有广泛的应用，如关键基础设施的故障检测、网络物理系统的入侵检测和金融服务的欺诈检测。检测延迟被定义为异常事件发生和检测之间的时间差，对于许多实际应用来说是至关重要的。更长的探测延迟可能会导致灾难性的后果，如桥梁坍塌或数百万人断电。在更短的检测延迟下，可以及时采取补救行动或对策，显著减少故障、攻击、事故或灾害造成的损害。该项目将开发一种新的低延迟异常检测方法，在保持令人满意的检测精度的同时，将检测延迟降至最低。这与目前大多数异常检测技术不同，大多数异常检测技术只关注检测精度，很少或根本不关注检测延迟。所提出的低延迟异常检测方法可广泛应用于民用、工业、科学和军事领域，如发电厂、通信网络、监控、结构健康监测和金融交易等。拟议研究工作的成果可以显著缩短对异常事件的响应时间，从而将网络攻击、系统故障、欺诈活动或自然灾害造成的损害和经济损失降至最低。通过该项目开发的技术可以提高物理和网络空间的安全和保障，并促进美国的竞争力和经济发展。通过拟议的研究工作获得的专业知识将用于促进新课程教材和学生研究项目的开发，并从技术创新和社会影响的角度提升学生的学习体验。该项目的目标是开发具有不完善数据模型的低延迟异常检测方法。设计目标是在保持令人满意的检测精度的同时最小化检测延迟。低延迟异常检测面临的最大挑战之一是对检测过程中使用的数据进行准确建模。由于决策需要在最小延迟的情况下做出，因此可用于模型训练或模型选择的数据量极其有限，特别是对于异常事件产生的数据。认识到获取精确数据模型的最大困难，该项目旨在主动开发专门针对不完美数据模型的新检测方法。拟议的研究活动将从以下几个角度转变异常检测的研究。首先，使用检测延迟而不是检测精度作为主要设计度量可以显著减少检测异常事件所需的时间量，同时仍然通过附加的设计约束保持令人满意的检测精度。其次，低延迟检测算法是专门为具有不完美数据模型的系统设计的。通过使用真实模型和不完美模型之间的Kullback-Leibler发散，分析了模型不确定性对检测延迟的基本性能限制。分析结果用于指导参数和非参数低延迟算法的设计，为最坏情况下的时延提供理论保证。第三，新开发的理论和算法将应用于电机故障检测和智能电网的入侵检测，其中的设计是考虑到这些网络物理系统的独特挑战和机遇。

Anomaly detection has a wide range of applications, such as fault detection for critical infrastructure, intrusion detection for cyber-physical systems, and fraud detection for financial services. Detection delay, which is defined as the time difference between the occurrence and detection of an anomaly event, is critical for many practical applications. A longer detection delay might lead to catastrophic results, such as the collapse of a bridge or the loss of power to millions of people. With a shorter detection delay, remedial actions or countermeasures can be carried out in a timely manner to significantly reduce the damages caused by faults, attacks, accidents, or disasters. This project will develop a new paradigm of low-latency anomaly detection methods that can minimize the detection delay while maintaining satisfactory detection accuracy. This is different from most current anomaly detection techniques, which focus solely on detection accuracy with little or no attention given to detection delays. The proposed low-latency anomaly detection methods can be applied to a wide range of civil, industrial, scientific, and military applications, such as power plants, communication networks, surveillance, structure health monitoring, and financial transactions. Outcomes of the proposed research work can significantly reduce the response time to anomaly events, thus minimizing the damages and economic losses caused by cyber-attacks, system failures, fraudulent activities, or natural disasters. Technologies developed through this project can improve the safety and security in both the physical and cyber-space, and promote the competitiveness and economic development of the United States. Expertise gained through the proposed research work will be used to facilitate the development of new course materials and student research projects, and enhance students' learning experiences from the perspectives of both technology innovations and social impacts.The goal of this project is to develop low-latency anomaly detection methods with imperfect data models. The design objective is to minimize the detection delay while maintaining satisfactory detection accuracy. One of the most formidable challenges faced by low-latency anomaly detection is the accurate modeling of the data used during detection. Since a decision needs to be made with minimum delay, there is extremely limited amount of data that can be used for model training or model selection, especially for data generated by the anomaly events. In recognition of the paramount difficulty in obtaining the precise data models, this project aims to proactively develop new detection methods tailored specifically for imperfect data models. The proposed research activities will transform the research on anomaly detection from the following perspectives. First, using detection delay instead of detection accuracy as the primary design metric can significantly reduce the amount of time required to detect an anomaly event, while still maintaining satisfactory detection accuracy through additional design constraints. Second, the low-latency detection algorithms are designed specifically for systems with imperfect data models. The fundamental performance limits imposed by model uncertainty on the detection latency are analytically characterized by using the Kullback-Leibler divergence between the true and imperfect models. The analytical results are used to guide the design of parametric and non-parametric low-latency algorithms, which can provide a theoretical guarantee on the worst case delay. Third, the newly developed theories and algorithms will be applied to the fault detection of electrical machines and the intrusion detection for smart grids, where the designs are performed by considering the unique challenges and opportunities of these cyber-physical systems.