Distance-Based Analysis for Complex High-Dimensional Data

在整个世纪，距离在统计学的重要领域发挥了重要作用，包括分类，聚类，判别分析，多维尺度，采样，空间统计，评分规则和机器学习中的核方法。距离也是定义发散测度、相对熵和信息增益的核心，其中一些是C.R.概念的基础。拉奥二次熵与生态学和其他科学领域的多样性分析。然而，目前有显着的差距，在我们的知识和新兴的统计文献中使用基于距离的测试和分析复杂的高维数据。 一个例子是相似性分析，这是最常引用和最广泛使用的基于距离的统计方法之一，但由于缺乏相关的数学知识而受到限制。这项研究将获得关于各种基于距离的统计方法的新的数学知识，并将其应用于回答林业、生态学和海洋科学的一些学科中出现的重要科学问题，例如：（1）热带森林中的生物多样性如何变化？(2)细菌群落的分类和功能特征如何随不同海洋区域的环境条件而变化？该项目建立了多个学科之间和两个美国学术机构之间的合作，并为研究生和本科生提供研究和培训。该项目开发了一套新的基于距离的统计方法和计算知识体系，用于分析以组合物，树，图形或网络形式出现的复杂，高维数据。这里考虑的距离都是非欧几里德的--要么是不满足任何三角不等式的非度量相异度，要么只是离散数--但它们都来自条件正定核。距离的示例包括平方欧几里德距离、Bray-Curtis相异度、Jensen-Shannon距离、Unifrac或Kantorovich-Rubinstein度量、Aitchison距离、编辑距离、各种图核和谱距离以及基于最优传输问题的其他距离。具体来说，该项目通过计算数据点的合并样本的最短哈密顿路径（或循环）的运行沿着，推进了精确的无分布的两个和多个样本运行测试，变化点和其他相关问题的数学理论和计算。该项目还考虑了相似性分析和相关的基于距离的排名测试，并得出新的数学结果，使我们能够进行更先进的统计分析。该项目有助于：㈠更深入地分析热带森林的生物多样性; ㈡调查原核生物群落的分类和功能概况如何随不同海洋区域的环境条件而变化; ㈢研究太平洋深海泥中稀土元素组成的可变性;和（iv）了解俄勒冈州海岸潮间带群落与上升流和营养物输送的关系。该项目整合了数学研究、科学和教育，并将为研究生提供论文工作的机会。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Throughout the course of the twentieth century, distances have played a significant role in important areas of statistics, which include classification, clustering, discriminant analysis, multidimensional scaling, sampling, spatial statistics, scoring rules, and kernel methods in machine learning. Distances are also central to the definition of divergence measures, relative entropy and information gain, some of which are fundamental to the concept of C.R. Rao's quadratic entropy and the analysis of diversity in ecology and other areas of science. Yet, at present there are significant gaps in our knowledge and in the emerging statistical literature on the use of distance-based tests and analyses for complex high dimensional data. One example is analysis of similarity which is among the most cited and most widely used distance-based statistical methods but is limited by an absence of relevant mathematical knowledge. This research will derive new mathematical knowledge on various distance-based statistical methods, and apply this for providing answers to important scientific questions arising in a number of disciplines in forestry, ecology and marine science, such as: (1) how biodiversity changes in tropical forests? (2) how taxonomic and functional profiles of bacterial communities change with environmental conditions in different oceanic regions? The project establishes collaborations among several disciplines and between two US academic institutions and provides research and training to graduate and undergraduate students. The project develops a new body of knowledge on distance-based statistical methods and computation for analyzing complex, high dimensional data that arise in the form of compositions, trees, graphs, or networks. The distances considered here are all non-Euclidean -- either non-metric dissimilarities that do not satisfy any triangular inequalities or just discrete numbers -- but they all arise from conditionally positive definite kernels. Examples of distances include the squared Euclidean distance, the Bray-Curtis dissimilarity, the Jensen-Shannon distance, Unifrac or the Kantorovich-Rubinstein metric, the Aitchison distance, the edit distance, various graph kernel and spectral distances, and other distances based on optimal transport problems. Specifically, the project advances the mathematical theory and computation of exact distribution-free two and multi-sample runs tests, change points, and other related problems by counting runs along the shortest Hamiltonian path (or loop) of the pooled sample of data points. The project also considers analysis of similarity and related distance-based rank tests and derives new mathematical results that allow us to pursue more advanced statistical analyses. The project contributes to: (i) a deeper analysis of biodiversity in tropical forest; (ii) an investigation of how taxonomic and functional profiles of prokaryotic communities change with environmental conditions in different oceanic regions; (iii) a study of the variability of composition of rare earth elements in deep-sea muds of the Pacific Ocean; and (iv) an understanding of the relationship of intertidal communities in the Oregon coast with respect to upwelling and nutrient delivery. The project integrates mathematics research, science and education and will provide opportunities for dissertation work for graduate students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.