Nonparametric Outlyingness and Descriptive Measures in Multivariate and General Data Settings

该项目扩展了核心统计科学中两个密切互动的领域的基础：非参数离群值识别和稳健的描述性测量。强调多变量和更复杂的数据类型。除主体以外的数据点(“离群值”)可能对统计分析产生不利影响，除非查明并加以考虑。这一关切是在新的背景下产生的，需要更新和扩大现有方法的表述。除了位置和偏度外，使用重尾数据和偏度和峰度描述性指标的多变量建模涉及对异常值的更多关注。多种新数据类型(功能、形状、图像、集合、符号、传感器、流、树、图形等)面对复杂但特别的计算机科学，数据挖掘方法需要更系统的处理。计算几何中的形状拟合问题带来了新形式的离群点问题。这项研究开发了新的一般基本方法来检测异常值，消除了对只处理异常值而不实际识别输入空间中的异常值的算法的依赖，消除了对椭圆异常轮廓的过度依赖，并加强了对重尾数据的适应。该项目的总体目标是为异常值检测建立扩展的概念统计学基础，并为位置、离散度、偏度、峰度等稳健的描述性度量开发新的结构，目的是在一般数据环境中广泛应用。随着计算资源的不断增加，统计数据分析和建模的范围正在扩大，以适应紧迫的新应用领域。科学和工程的所有领域的数据都具有复杂的多维结构，通常具有大样本大小，涉及曲线、图像、文本和其他对象，通常位于数据流或网络结构中。这在检测和处理“异常”数据点(“离群值”)方面产生了新的重大问题。哪些箱子是分开的？这些“不寻常”的案例如何影响对整个数据集的统计分析？当数据海量且涉及许多变量时，怎样的计算步骤才能有效地找到离群值？哪些一般原则适用于各种新情况，如欺诈检测、入侵检测、网络分析和数据挖掘？如何定义与几个相关数据集(例如图像、文本和传感器数据)的融合相关的“离群值”，就像国土安全部中可能出现的那样？这项研究解决了这些基本的实际问题，目的是在既定统计原则的基础上，合理地发展新的方法。

This project extends foundations in two closely interactive areas of core statistical science: nonparametric outlier identification, and robust descriptive measures. Multivariate and more complex data types are emphasized. Data points apart from the main body ("outliers") can adversely affect statistical analyses unless identified and taken into account. This concern is arising in new contexts calling for updated and broadened formulations of current methods. Multivariate modeling with heavy tailed data and with skewness and kurtosis descriptive measures in addition to location and skewness involves increased concern with outliers. Diverse new data types (functional, shape, image, set, symbolic, sensor, stream, tree, graph, etc.) being treated by sophisticated but ad hoc computer science data mining approaches need more systematic treatment. Shape-fitting problems in computational geometry impose new forms of outlier issues. The study develops new general foundational approaches to outlier detection, eliminates reliance on algorithms that only handle outliers without actually identifying them in the input space, eliminates undue reliance upon elliptical outlyingness contours, and strengthens the accommodation of heavy tailed data. The overall project goals are to establish extended conceptual statistical foundations for outlier detection and to develop new structures for robust descriptive measures of location, dispersion, skewness, kurtosis, etc., with the aim of broad application across general data settings.With advancing computational resources, the scope of statistical data analysis and modeling is widening to accommodate pressing new arenas of application. Data in all areas of science and engineering has complex multidimensional structure, typically with large sample sizes and involving curves, images, text, and other objects, often within astream or network structure. This is generating major new problems in detection and handling of "anomalous" data points ("outliers"). Which cases stand apart? How do the "unusual" cases impact statistical analyses on the full data set? What computational steps efficiently find the outliers when the data is massive and involves many variables? What general principles apply across diverse new situations such as fraud detection, intrusion detection, network analysis, and data mining? How to define "outlier" relative to a fusion of several related data sets, for example image, text, and sensor data, as might arise in Homeland Security? This study addresses these basic practical questions with the aim of developing new methodological approaches soundly based upon established statistical principles.