ACT SGER: Locating Sparse Events in High Speed Stream Data, with a Focus on Statistical Analysis

PI建议研究在识别高速流序列中的稀疏事件时产生的统计问题。流数据在许多应用中经常遇到。例子包括(1)互联网定价和网络安全中的互联网流量数据，(2)通信网络中的海量信息和(恐怖主义)活动监控的需求。这些问题与“打击恐怖主义的方法”这一主题密切相关。这些问题有一些共同的特点：感兴趣的事件非常稀疏；识别需要以非常快的速度实现。此外，在网络安全和恐怖活动监控中，错误检测远远比错误警报更具灾难性。如何找到高效的方法来高速识别流数据中的稀疏事件，这对统计学家和计算科学家都提出了巨大的挑战。采购经理人建议研究上述问题，侧重于统计建模方面，即解决这些问题的最佳建模和计算策略是什么。流数据面临的主要挑战是快速实时算法的发展。统计学家已经在序贯分析中开发了强大的工具，例如MCMC和其他基于最大似然的方法。与此同时，提出的问题在统计上也没有得到太大的重视。需要进行新的分析，将数理统计的最佳性和科学计算的效率结合起来。PI将从两个关键思想开始：基于树的匹配方法和特殊的决策树级联。他们将受益于与计算机视觉和计算机安全等其他领域的研究人员的互动。这项支助将使私人投资主任能够培训两名研究生从事这一领域的工作。这一支持还将有助于启动在流数据中计算效率高的统计建模方面的更大努力。PIS将为他们的算法开发软件，并在互联网上提供。该奖项由NSF和情报界共同支持。数学和物理科学局的反恐方法方案支持基础研究和劳动力发展方面的新概念，有可能对国家安全作出贡献。

The PIs propose to study statistical problems that arise from identifying sparse events in high-speed stream sequences. Stream data are commonly encountered in many applications. Examples include (1) internet traffic data in internet pricing and network security, (2) the vast amount of information in communication networks and the demand of (terrorism) activity monitoring. These problems are closely related to the theme of Approaches to Combat Terrorism (ACT). These problems share some common properties: the events of interests are 'extremely' sparse; and the identification needs to be realized at a 'very' high speed. Moreover, in both network security and terrorist activity monitoring, false detection is far more disastrous than false alarms. It poses big challenges to both statisticians and computational scientists to find efficient methods to identify sparse events in stream data at high speed. The PIs propose to study the above problems, emphasizing the statistical modeling aspect, namely, what is the optimal modeling and computational strategy to solve these problems. The main challenge in stream data is the development of fast real time algorithms. Statisticians have developed powerful tools in sequential analysis, e.g., MCMC and other maximal likelihood based approaches. At the same time, the proposed problem has not been given much attention in statistics. New analysis needs to be done to integrate both optimality in mathematical statistics and efficiency in scientific computing. The PIs will start with two key ideas: a tree-based matching method and a special decision tree - cascade. They will benefit from interactions with researchers in other fields such as computer vision and computer security. The support will allow the PIs to train two graduate students to work in this area. The support will also help to jump start a larger effort on computationally efficient statistical modeling in stream data. The PIs will develop software for their algorithms and make it available over the internet.This award is supported jointly by the NSF and the Intelligence Community. The Approaches to Combat Terrorism Program in the Directorate for Mathematical and Physical Sciences supports new concepts in basic research and workforce development with the potential to contribute to national security.