Geometry Aware Exploratory Data Analysis and Inference Methods for Complex Data

复杂的大数据经常出现在科学领域，并已成为当代数据科学的标准。众所周知，分析存在于度量空间中的数据是困难的，缺乏基本的向量空间操作，如加法和标量乘法，并且数据元素之间没有排序。这些数据以直方图、网络、图像、系统发育树等样本的形式出现，并出现在许多领域，如健康监测、神经科学、商业和经济研究、气候和环境研究、进化遗传学、社会科学和人口统计学。当观察到的复杂数据是动态的时，例如，当数据是时变的或在其他连续域上观察时，挑战会被放大。该项目将通过创建一个理论上合理且用户友好的实用工具包来推动现代数据分析技术的前沿，该工具包将克服几项重要数据分析任务的这些挑战。这些新方法仅基于数据元素之间的成对距离，并且通过设计进行自由调整，将立即满足科学家和工程师使用不同数据表示的需求，例如纵向fMRI研究，病毒突变的在线检测，了解微生物多样性组成，监测电子健康分析中的每日血糖分布，时变基因调控网络，了解社会进化趋势等等，为从业者提供了一系列现成的工具，在进入下游建模任务之前对复杂数据进行探索性分析。该奖项还将支持研究生的培训，并为本科生提供研究机会。无模型的基于距离的方法推动了面向复杂非欧几里德数据的统计方法的成功开发，对环境数据空间或数据分布的要求最低。本研究旨在通过为常见的数据分析工作开发新的严格合理的算法，并建立位于统计学核心的推理程序，并构成大多数科学家试图用数据回答的基础，来扩展对象数据分析的方法论。为了解决对象数据中缺乏矢量空间结构和数据元素之间缺乏排序的关键挑战，新的发展将基于深度剖面的概念，深度剖面是由数据定律决定的距离分布，以及传输等级，是使用深度剖面之间的最佳传输图构建的对象数据的中心向外排序方案。具体的子项目将集中在基于秩的对象数据聚类和分类，离群点检测和模式为中心的数据分析程序。具有严格理论的推理框架将被设计用于新颖的双样本测试、独立性测试、变点检测和定位，所有这些都将是基于距离的并且易于实现。最后，新的工具将扩大到包括时变对象数据的探索性分析和降维，无论是在观测时间密集的情况下，还是在更具有挑战性的情况下，只有稀疏的时间测量是不规则的观测。理论和方法的发展将涉及从实证过程和U-过程理论，M-估计和功能数据分析的工具。高效和可扩展的软件实现以及吸引人的可视化代码，这对对象数据来说是极具挑战性的，将免费提供给从业者。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Complex big data appear routinely in the sciences and has become standard fare in contemporary data science. It is known to be difficult to analyze data that live in metric spaces, lacking fundamental vector space operations like addition and scalar multiplication and with no ordering between the data elements. Such data show up in the form of samples of histograms, networks, images, phylogenetic trees, and so on, and in multitudes of fields such as health monitoring, neuroscience, business and economics research, climate and environmental studies, evolutionary genetics, social sciences, and demography. Challenges are magnified when the observed complex data are dynamic, for example, when the data are time-varying or observed on other continuous domains. This project will push the frontiers in the state of the art of modern data analysis by creating a theoretically sound and user-friendly practical toolkit that will overcome these challenges for several important data analysis tasks. The new methods, being rooted only in pairwise distances between the data elements and tuning free by design, will immediately cater to the needs of scientists and engineers working with diverse representations of data, for example in longitudinal fMRI studies, online detection of the mutations in the virus phylogeny, understanding microbial diversity compositions, monitoring daily blood glucose distributions in electronic health analytics, time-varying gene-regulatory networks, understanding trends in social evolution and many more, offering practitioners a bundle of off-the-shelf tools to carry out exploratory analysis on the complex data before moving on to the downstream modeling tasks. The award will also support graduate students' training and offer research opportunities to undergraduates.Model-free distance-based approaches drive the success of developing statistical methods oriented to complex non-Euclidean data with minimal requirements on the ambient data space or the data distribution. This research aims to expand the arsenal of methodology in object data analysis by developing new rigorously justified algorithms for common data analysis jobs and building inference procedures that lie at the heart of statistics and constitute the basis of what most scientists attempt to answer with data. To address the key challenge of the lack of a vector space structure in object data and the absence of ordering among the data elements, the new developments will be based on the concepts of depth profiles, which are the distributions of distances as dictated by the law of the data, and the transport ranks, that are center-outward ordering schemes for object data constructed using optimal transport maps between the depth profiles. Specific sub-projects will focus on rank-based object data clustering and classification, outlier detection, and mode-centric data analysis procedures. Inferential frameworks with rigorous theory will be designed for novel two-sample tests, independence tests, change point detection, and localization, all of which will be distance-based and easily implementable. Finally, the new tools will be broadened to include exploratory analysis and dimension reduction for time-varying object data, both when the observations are dense in time and the more challenging case when only sparse measurements in time are observed irregularly. Theory and methodology development will involve tools from the empirical process and U-process theory, M-estimation, and functional data analysis. Efficient and scalable software implementations together with codes for appealing visualizations, which are extremely challenging for object data, will be made freely available for practitioners.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.