BIGDATA:F: Statistical and Computational Optimal Transport for Geometric Data Analysis

目前大数据的方法和伴随的计算方法已经留下了关键的应用程序，其中数据不是单个点的集合，而是整个几何对象。这些应用包括医学成像、自动驾驶汽车的激光雷达和单细胞RNA测序，仅举几例。要将更简单的数据处理和统计技术的压倒性成功转移到这一制度，不仅需要大型数据集，而且还需要适当的模型和算法来分析这类更一般的数据。由于在计算方面的最新进展，最优运输理论已被证明对解决这些限制很有价值。然而，将最优运输理解为一种统计工具仍处于初级阶段。本项目旨在开发一个基于最优传输的“几何数据分析”工具箱来处理这些新的数据集。这项提议将有助于创建一种跨学科互动和协作的共同语言。这项研究的大部分将被整合到本课程中，并通过麻省理工学院开放式课程软件提供。这一建议还将使博士生和本科生能够进行丰富的跨学科培训。建议的方法建立在丰富的最优运输(OT)数学理论基础上。除了严格的统计和计算分析外，这一理论还为开发新的几何数据分析方法提供了一个框架。计算最优传输的新生理论仍然在很大程度上脱离了统计学，许多方法没有适当地考虑采样和测量噪声。为了避免过度拟合的陷阱，这一建议独特而系统地采用了统计方法来分析几何数据。通过了解OT用于统计建模的理论优势和不足，它将导致具有强大统计保证的可扩展OT算法。这一提议的一个具体结果是形成了一个连贯的工具箱，不仅扩展了平均，还扩展了经典统计中的回归、分类、聚类和其他概念，以一种捕捉数据全局几何特征的方式。它不仅将对医学图像的分析产生直接影响，还将对LiDAR为自动驾驶汽车收集的点云、单细胞RNA测序产生的基因表达序列以及其他各种但大规模的数据源的各种应用产生直接影响。这些数据集包含数百万个实体，但拒绝应用标准的统计程序；目前对其进行分析的最先进技术是临时的，不能推广，无法达到其他领域“大数据”工具所达到的质量。通过将这项工作纳入麻省理工学院统计的新学位项目，将产生教育影响。该奖项反映了NSF的法定使命，并已通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Current approaches to big data and accompanying computational methods have left behind critical applications where the data is not a collection of individual points, but rather whole geometric objects. Such applications include medical imaging, LiDAR for self-driving cars, and single-cell RNA sequencing, to name a few. Transferring the overwhelming success of simpler data processing and statistical techniques to this regime requires not only large datasets, but also suitable models and algorithms for analysis of this more general type of data. The theory of optimal transport has proven valuable to address these limitations thanks to recent advances on the computational front. Yet, understanding optimal transport as a statistical tool is still in its infancy. This project aims at developing a "geometric data analysis" toolbox based on optimal transport to tackle these new datasets. This proposal will help create a common language to interact and collaborate across disciplines. Much of this research will be integrated in this curriculum and made available through MIT OpenCourseWare. This proposal will also enable rich interdisciplinary training of PhD and undergraduate students.The proposed methods are built around the rich mathematical theory of optimal transport (OT). This theory provides a framework for the development of new methods for geometric data analysis in addition to their rigorous statistical and computational analysis. The nascent theory of computational optimal transport is still largely dissociated from statistics, and many methods do not account properly for sampling and measurement noise. To avoid the pitfalls of overfitting, this proposal singularly and systematically takes a statistical approach to geometric data analysis. With an understanding of the theoretical advantages and drawbacks of OT for statistical modeling, it will lead to scalable OT algorithms with strong statistical guarantees. A tangible outcome of this proposal is a cohesive toolbox extending not only averaging but also regression, classification, clustering, and other notions from classical statistics in a fashion that captures global geometric features of data. It will have a direct impact on various applications in analysis of not only medical images but also point clouds gathered by LiDAR for self-driving cars, sequences of gene expressions produced by single-cell RNA sequencing, and other diverse yet large-scale sources of data. These datasets contain millions of entities but resist application of standard statistical procedures; current state-of-the-art techniques for their analysis are ad-hoc, not generalizable, and fail to reach the quality achieved by "big data" tools in other domains. Educational impact will be made by incorporating this work in new degree programs in statistics at MIT.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.