BIGDATA:F: Statistical and Computational Optimal Transport for Geometric Data Analysis

目前的大数据方法和相应的计算方法已经落后于关键应用程序，其中数据不是单个点的集合，而是整个几何对象。这些应用包括医学成像、用于自动驾驶汽车的激光雷达和单细胞RNA测序等。将简单的数据处理和统计技术的巨大成功转移到这一制度不仅需要大型数据集，而且还需要用于分析这种更一般类型的数据的合适模型和算法。最佳运输理论已被证明是有价值的，以解决这些限制，由于最近的进展，在计算方面。然而，将最佳运输理解为一种统计工具仍处于起步阶段。该项目旨在开发一个基于最佳运输的“几何数据分析”工具箱，以处理这些新的数据集。该提案将有助于创建跨学科互动和协作的共同语言。这些研究的大部分内容将被整合到本课程中，并通过麻省理工学院的开放式课程提供。该建议还将使博士和本科生的丰富的跨学科培训。所提出的方法是建立在丰富的数学理论的最佳运输（OT）。这一理论提供了一个框架，除了严格的统计和计算分析的几何数据分析的新方法的发展。计算最优传输的新生理论在很大程度上仍然与统计无关，许多方法不能正确考虑采样和测量噪声。为了避免过度拟合的陷阱，该建议单独和系统地采用统计方法进行几何数据分析。了解OT在统计建模中的理论优势和不足，将有助于开发具有强统计保证的可扩展OT算法。这一提议的一个切实成果是一个有凝聚力的工具箱，它不仅扩展了平均值，而且还扩展了回归、分类、聚类和经典统计学中的其他概念，其方式是捕捉数据的全局几何特征。 它将对各种应用产生直接影响，不仅包括医学图像分析，还包括用于自动驾驶汽车的LiDAR收集的点云，单细胞RNA测序产生的基因表达序列以及其他多样化但大规模的数据来源。 这些数据集包含数百万个实体，但无法应用标准统计程序;目前最先进的分析技术是临时的，不可推广，无法达到其他领域的“大数据”工具所达到的质量。 通过将这项工作纳入麻省理工学院的统计学新学位课程，将产生教育影响。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Current approaches to big data and accompanying computational methods have left behind critical applications where the data is not a collection of individual points, but rather whole geometric objects. Such applications include medical imaging, LiDAR for self-driving cars, and single-cell RNA sequencing, to name a few. Transferring the overwhelming success of simpler data processing and statistical techniques to this regime requires not only large datasets, but also suitable models and algorithms for analysis of this more general type of data. The theory of optimal transport has proven valuable to address these limitations thanks to recent advances on the computational front. Yet, understanding optimal transport as a statistical tool is still in its infancy. This project aims at developing a "geometric data analysis" toolbox based on optimal transport to tackle these new datasets. This proposal will help create a common language to interact and collaborate across disciplines. Much of this research will be integrated in this curriculum and made available through MIT OpenCourseWare. This proposal will also enable rich interdisciplinary training of PhD and undergraduate students.The proposed methods are built around the rich mathematical theory of optimal transport (OT). This theory provides a framework for the development of new methods for geometric data analysis in addition to their rigorous statistical and computational analysis. The nascent theory of computational optimal transport is still largely dissociated from statistics, and many methods do not account properly for sampling and measurement noise. To avoid the pitfalls of overfitting, this proposal singularly and systematically takes a statistical approach to geometric data analysis. With an understanding of the theoretical advantages and drawbacks of OT for statistical modeling, it will lead to scalable OT algorithms with strong statistical guarantees. A tangible outcome of this proposal is a cohesive toolbox extending not only averaging but also regression, classification, clustering, and other notions from classical statistics in a fashion that captures global geometric features of data. It will have a direct impact on various applications in analysis of not only medical images but also point clouds gathered by LiDAR for self-driving cars, sequences of gene expressions produced by single-cell RNA sequencing, and other diverse yet large-scale sources of data. These datasets contain millions of entities but resist application of standard statistical procedures; current state-of-the-art techniques for their analysis are ad-hoc, not generalizable, and fail to reach the quality achieved by "big data" tools in other domains. Educational impact will be made by incorporating this work in new degree programs in statistics at MIT.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.