Automated Statistical Techniques for Systematic Anomaly Detection in High Frequency Data

点击翻译按钮获取中文摘要

Smart and intelligent processes have become common in business, industry, and other areas. Among the tasks are the design, development, and maintenance of automated and autonomous systems, processing massive datasets, and facilitating real-time decision making. Naturally, such processes are of immense interest to competitors, and thus become increasingly vulnerable to cyber attacks, intrusions, theft, and other disruptions. Major research and development areas that arise in this context are risk assessment and classification, their timely detection and mitigation, system vulnerability and reliability assessment to specific risks, and resilience development. The extensive knowledge and expertise that I have acquired over the many years of relevant practical and academic work in these areas have given rise to the proposed research program. In a nutshell, the program is about distinguishing true signals from the noise, the latter being a generic term for anomalies, aberrations, intrusions, and other exogenous disruptions. That is, the proposed research program is designed to advance anomaly-detection methods and with them associated statistical inference techniques, transition them into practice, facilitate timely anomaly detection and their effective mitigation strategies. Although fully automated methods for anomaly detection are not possible, we shall nevertheless aim at developing self-learning (e.g., unsupervised training, machine- and deep-learning) algorithms to sequentially increase the level of automation and efficiency. The program is highly multifaceted. It includes the development of theoretical (statistical, mathematical, etc.) foundations, extensive computer coding and calibration, as well as practical implementation, keeping in mind that each application requires specialized tuning of the anomaly-detection method. Many undergraduate and graduate students, as well as postdoctoral fellows and research associates, will be engaged in the project.