Data Management and Analytics System for Online Flight Conformance Monitoring and Anomaly Detection
Data Management and Analytics System for Online Flight Conformance Monitoring and Anomaly Detection
复制标题
DOI：
10.1145/3347146.3359378
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
S. Ayhan;H. Samet
中科院分区：
文献类型：
--
作者：
S. Ayhan;H. Samet
世界各地的空中导航服务提供商 (ANSP) 一直在努力开发更好的方法来监控计划航线的符合情况并检测特定空域内的异常情况。一致性监控和异常检测对于从战略和战术上更好地管理空域至关重要，从而实现更高水平的自动化，从而减少空中交通管制员的工作量。尽管现有的静态空中交通数据量有限的方法已经能够在一定程度上解决该问题，但对不断增加的大量流式空中交通数据的高速在线一致性监控和异常检测的数据管理和查询处理仍然是一个挑战。在本文中，我们提出了一种新颖的数据管理和分析系统，可以持续监控航班的一致性并准确检测国家空域系统（NAS）内的异常情况。传入的交通流管理 (TFM) 数据是流式的、庞大的、不相关的且有噪声的数据。在整个数据管道中，系统分 3 个步骤监控航班并检测异常：在预处理步骤中，系统持续处理传入的原始航班数据，并使其可用于下一步，即创建和维护临时键值数据存储以进行高效的查询处理。最后一步，系统从历史轨迹和相关天气参数中学习，并构建长短期记忆 (LSTM) 模型。随着飞行的进行，作为实时数据流一部分的不合格轨迹段会被视为异常。对美国真实空中交通和天气数据的评估验证了我们的系统能够有效、准确地检测异常情况。
Air Navigation Service Providers (ANSP) worldwide have been making a considerable effort for the development of a better method to monitor conformance to the planned routes and detect anomalies within a particular airspace. Conformance monitoring and anomaly detection are crucial for a better managed airspace, both strategically and tactically, yielding a higher level of automation and thereby reducing the air traffic controller's workload. Although the prior approaches with limited amount of static air traffic data have been able to address the problem to some extent, data management and query processing of ever-increasing vast volume of streaming air traffic data at high rates for online conformance monitoring and anomaly detection still remain a challenge. In this paper, we present a novel data management and analytics system to continuously conformance monitor flights and accurately detect anomalies within the National Airspace System (NAS). The incoming Traffic Flow Management (TFM) data is streaming, big, uncorrelated and noisy. In the overall data pipeline, the system monitors flights and detects anomalies in 3 steps: In the preprocessing step, the system continuously processes the incoming raw flight data and makes it available for the next step where an interim Key-Value data store is created and maintained for efficient query processing. In the final step, the system learns from historical trajectories and pertinent weather parameters and builds a Long Short-Term Memory (LSTM) model. As the flights progress, the non-conforming trajectory segments as part of the live data stream are raised as anomalies. Evaluations on real air traffic and weather data in the U.S. verify that our system efficiently and accurately detects anomalies.