ATD: Online Multiscale Algorithms for Geometric Density Estimation in High-Dimensions and Persistent Homology of Data for Improved Threat Detection

研究人员和他的同事们开发了新的想法来应对威胁检测中的挑战。出发点是从高维数据的多尺度几何和拓扑分析的见解：低内在维数，流形结构和/或其他类型的几何属性的数据是利用他们的新方法的任务，如密度估计，异常检测，降维和分类。这种方法的优点是能够适应数据的低固有维度，从而导致算法有效地执行这些任务，无论是在需要学习的样本量方面，还是在计算成本方面，都导致新一代的结果和算法。他们的研究重点是检测化学攻击，这是最致命的威胁之一，特别是用于化学检测的高光谱成像，特别是使用长波HSI系统中内置的大气长波红外光谱。他和他的合作者将这些技术应用于HSI数据，以图像和包含化学羽流的HSI电影的形式，利用所提出的技术的速度。输入数据（图像，光谱等）对于许多威胁检测问题来说，通常是大的、高维的、被噪声破坏的，并且经常由于环境条件而受到失真。许多威胁检测任务属于以下几大类之一：回归、分类、异常或离群值检测以及变点检测。这些任务面临着基本的维数灾难：为了达到目标精度水平，所需的观测数量在数据的维数中呈指数级增长。这样的维度可以是感兴趣的子图像中的像素的数量或者高光谱图像（HSI）或光谱仪中的光谱带的数量，并且可以非常大。这使得高维数据的分析变得毫无希望，除非我们能发现数据的低维表示，或者至少是那些足以执行手头任务的数据特征：PI和他的同事开发了新的技术来发现这种表示，并利用它们来建模数据，并检测不断变化的数据中的异常。这些构造和算法增强了我们的威胁检测能力，并且是在威胁检测中产生的大型数据集分析领域推进信息技术的关键。

The investigator and his colleagues develop novel ideas to tackle challenges in threat detection. The starting point are insights from multiscale geometric and topological analysis of high-dimensional data: low-intrinsic dimensionality, manifold structures and/or other types of geometric properties of the data are exploited by their novel approaches for tasks such as density estimation, anomaly detection, dimensionality reduction and classification. This approach has the advantage of being adaptive to the low intrinsic dimensionality of the data, thereby leading to algorithms to perform these tasks efficiently, both in terms of sample size require to learn, and in terms of computational costs, leading to a new generation of results and algorithms. Their research focuses on the detection of chemical attacks, which are one of the most pernicious threats, and in particular on hyperspectral imaging for chemical detection, specifically using atmospheric longwave infrared spectroscopy built into the longwave HSI systems. He and his collaborators apply these techniques to HSI data, in the form of images and streaming HSI movies containing chemical plumes, taking advantage of the speed of the proposed techniques.The input data (images, spectra, etc...) for many threat detection problems is typically large, high-dimensional, corrupted by noise, and often subject to distortions due to environmental conditions. Many threat detection tasks fall into one of the following broad categories: regression, classification, anomaly or outlier detection, and changepoint detection. These tasks face the fundamental curse of dimensionality: to achieve a target level of accuracy, the number of observations required is exponential in the number of dimensions of the data. Such dimension may be the number of pixels in a sub-image of interest or the number of spectral bands in a HyperSpectral Image (HSI) or a spectrometer, and may be very large. This makes the analysis of high-dimensional data hopeless unless we can discover a low-dimensional representation of the data, or at least of those features of the data that are sufficient to perform the task at hand: the PI and his colleagues develop novel techniques for discovering such representations and exploiting them to model the data, and detecting anomalies in evolving data. These constructions and algorithms enhance our capability in threat detection, and are key to advance information technology in the field of analysis of large data sets arising in threat detection.