Automated Statistical Techniques for Systematic Anomaly Detection in High Frequency Data

智能流程已在商业、工业和其他领域变得普遍。其中的任务包括设计、开发和维护自动化和自治系统，处理海量数据集，并促进实时决策。当然，竞争对手对此类流程非常感兴趣，因此越来越容易受到网络攻击、入侵、盗窃和其他中断的影响。在这方面出现的主要研究和开发领域是风险评估和分类、及时检测和缓解、针对特定风险的系统脆弱性和可靠性评估以及复原力发展。多年来，我在这些领域的相关实践和学术工作中获得了广泛的知识和专业知识，这是拟议的研究计划的起因。简而言之，该程序是关于区分真实信号和噪声，后者是异常、异常、入侵和其他外部干扰的通用术语。也就是说，拟议的研究计划旨在推进异常检测方法和与之相关的统计推理技术，将其转化为实践，促进及时的异常检测及其有效的缓解策略。虽然不可能有完全自动化的异常检测方法，但我们将致力于开发自学习(例如，无监督训练、机器和深度学习)算法，以顺序地提高自动化水平和效率。该计划是高度多面性的。它包括理论(统计、数学等)的发展。基础、广泛的计算机编码和校准以及实际实施，铭记每一项应用都需要对异常检测方法进行专门调整。许多本科生和研究生，以及博士后研究员和研究助理，将参与该项目。

Smart and intelligent processes have become common in business, industry, and other areas. Among the tasks are the design, development, and maintenance of automated and autonomous systems, processing massive datasets, and facilitating real-time decision making. Naturally, such processes are of immense interest to competitors, and thus become increasingly vulnerable to cyber attacks, intrusions, theft, and other disruptions. Major research and development areas that arise in this context are risk assessment and classification, their timely detection and mitigation, system vulnerability and reliability assessment to specific risks, and resilience development. The extensive knowledge and expertise that I have acquired over the many years of relevant practical and academic work in these areas have given rise to the proposed research program. In a nutshell, the program is about distinguishing true signals from the noise, the latter being a generic term for anomalies, aberrations, intrusions, and other exogenous disruptions. That is, the proposed research program is designed to advance anomaly-detection methods and with them associated statistical inference techniques, transition them into practice, facilitate timely anomaly detection and their effective mitigation strategies. Although fully automated methods for anomaly detection are not possible, we shall nevertheless aim at developing self-learning (e.g., unsupervised training, machine- and deep-learning) algorithms to sequentially increase the level of automation and efficiency. The program is highly multifaceted. It includes the development of theoretical (statistical, mathematical, etc.) foundations, extensive computer coding and calibration, as well as practical implementation, keeping in mind that each application requires specialized tuning of the anomaly-detection method. Many undergraduate and graduate students, as well as postdoctoral fellows and research associates, will be engaged in the project.