Distance-Based Analysis for Complex High-Dimensional Data

在整个二十世纪的过程中，距离在统计学的重要领域发挥了重要作用，包括分类、聚类、判别分析、多维尺度、采样、空间统计、评分规则和机器学习的核心方法。距离也是定义发散度、相对熵和信息增益的核心，其中一些对于C.R.拉奥的二次熵概念以及生态学和其他科学领域的多样性分析是基本的。然而，目前在我们的知识和新兴的统计文献中，在使用基于距离的测试和分析复杂的高维数据方面存在重大差距。一个例子是相似性分析，它是被引用最多、使用最广泛的基于距离的统计方法之一，但由于缺乏相关的数学知识而受到限制。这项研究将从各种基于距离的统计方法中获得新的数学知识，并将其应用于为林业、生态学和海洋科学中的一些学科中出现的重要科学问题提供答案，例如：(1)热带森林中的生物多样性如何变化？(2)细菌群落的分类和功能分布如何随着不同海洋地区的环境条件而变化？该项目建立了几个学科之间以及两个美国学术机构之间的合作，并为研究生和本科生提供研究和培训。该项目开发了基于距离的统计方法和计算的新知识体系，用于分析以组合、树、图或网络的形式出现的复杂的高维数据。这里考虑的距离都是非欧几里得的--要么是不满足任何三角不等式的非度量不同点，要么是仅仅是离散数--但它们都来自条件正定核。距离的例子包括欧几里得距离的平方、Bray-Curtis相异度、Jensen-Shannon距离、Unifrac或Kantorovich-Rubinstein度量、Aitchison距离、编辑距离、各种图核和谱距离以及基于最优传输问题的其他距离。具体地说，该项目通过计算沿数据点集合样本的最短哈密顿路径(或循环)的运行，提出了精确分布无两个和多个样本运行测试、变化点和其他相关问题的数学理论和计算。该项目还考虑了相似性分析和相关的基于距离的等级测试，并得出了新的数学结果，使我们能够进行更高级的统计分析。该项目有助于：(1)更深入地分析热带森林的生物多样性；(2)调查原核生物群落的分类和功能剖面如何随不同海洋区域的环境条件而变化；(3)研究太平洋深海泥中稀土元素组成的变异性；(4)了解俄勒冈海岸潮间带群落与上升流和营养物质输送之间的关系。该项目将数学研究、科学和教育融为一体，并将为研究生提供论文工作的机会。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Throughout the course of the twentieth century, distances have played a significant role in important areas of statistics, which include classification, clustering, discriminant analysis, multidimensional scaling, sampling, spatial statistics, scoring rules, and kernel methods in machine learning. Distances are also central to the definition of divergence measures, relative entropy and information gain, some of which are fundamental to the concept of C.R. Rao's quadratic entropy and the analysis of diversity in ecology and other areas of science. Yet, at present there are significant gaps in our knowledge and in the emerging statistical literature on the use of distance-based tests and analyses for complex high dimensional data. One example is analysis of similarity which is among the most cited and most widely used distance-based statistical methods but is limited by an absence of relevant mathematical knowledge. This research will derive new mathematical knowledge on various distance-based statistical methods, and apply this for providing answers to important scientific questions arising in a number of disciplines in forestry, ecology and marine science, such as: (1) how biodiversity changes in tropical forests? (2) how taxonomic and functional profiles of bacterial communities change with environmental conditions in different oceanic regions? The project establishes collaborations among several disciplines and between two US academic institutions and provides research and training to graduate and undergraduate students. The project develops a new body of knowledge on distance-based statistical methods and computation for analyzing complex, high dimensional data that arise in the form of compositions, trees, graphs, or networks. The distances considered here are all non-Euclidean -- either non-metric dissimilarities that do not satisfy any triangular inequalities or just discrete numbers -- but they all arise from conditionally positive definite kernels. Examples of distances include the squared Euclidean distance, the Bray-Curtis dissimilarity, the Jensen-Shannon distance, Unifrac or the Kantorovich-Rubinstein metric, the Aitchison distance, the edit distance, various graph kernel and spectral distances, and other distances based on optimal transport problems. Specifically, the project advances the mathematical theory and computation of exact distribution-free two and multi-sample runs tests, change points, and other related problems by counting runs along the shortest Hamiltonian path (or loop) of the pooled sample of data points. The project also considers analysis of similarity and related distance-based rank tests and derives new mathematical results that allow us to pursue more advanced statistical analyses. The project contributes to: (i) a deeper analysis of biodiversity in tropical forest; (ii) an investigation of how taxonomic and functional profiles of prokaryotic communities change with environmental conditions in different oceanic regions; (iii) a study of the variability of composition of rare earth elements in deep-sea muds of the Pacific Ocean; and (iv) an understanding of the relationship of intertidal communities in the Oregon coast with respect to upwelling and nutrient delivery. The project integrates mathematics research, science and education and will provide opportunities for dissertation work for graduate students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.