CAREER: Robust Algorithms for Corrupted Data

近年来，数据驱动的科学发现和技术创新方法有所增加。规模庞大、种类繁多的数据集通常用于探索新现象和训练算法。然而，随着数据量和复杂性的增长，数据质量往往会下降。数据可能会受到噪声、离群值、缺失值和其他形式的信息丢失的困扰。该项目将设计强大，有效的算法来分析这种损坏的数据。这些方法将被部署在成像，生物学和其他高影响力领域的问题上。它们将在通用代码中实现，供公众使用。除了研究成果外，该项目还将创建教育资源，在数据分析的理论和实践方面培训高年级本科生和研究生。将考虑两种互补的方法来处理损坏的数据。对于第一类方法，将假设数据具有受加性噪声和线性滤波器扰动的底层低秩结构。从随机矩阵理论的新成果将开发信号恢复问题，在这种情况下，包括估计的协方差和距离矩阵。第二类方法将利用数据中的几何结构。将设计来自计算谐波分析的新方法，以通过揭示数据点之间的关系信息来学习数据集的几何形状，并将此几何信息用于聚类，回归和其他任务。这两种方法都有可能产生不仅在统计上稳健，而且计算效率高的方法。该奖项反映了NSF的法定使命，并被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Recent years have seen an increase in data-driven approaches to scientific discovery and technological innovation. Datasets of enormous size and variety are routinely used to explore new phenomena and train algorithms. As data volume and complexity grow, however, data quality often decreases. Data can be plagued by noise, outliers, missing values, and other forms of information loss. This project will design robust, efficient algorithms for analyzing such corrupted data. These methods will be deployed on problems in imaging, biology, and other high-impact domains. They will be implemented in general-purpose codes, to be released for public use. In addition to its research products, this project will also create educational resources to train students at the advanced undergraduate and beginning graduate level in the theory and practice of data analysis.Two complementary methodologies will be considered for processing corrupted data. For the first class of methods, the data will be assumed to have an underlying low-rank structure that is perturbed by both additive noise and linear filters. New results from random matrix theory will be developed for signal recovery problems in this setting, including the estimation of covariance and distance matrices. The second class of methods will exploit geometric structures in data. Novel methods from computational harmonic analysis will be devised to both learn the geometry of a dataset by uncovering relational information between data points, and to use this geometric information for clustering, regression, and other tasks. Both approaches have the potential to yield methods that are not only statistically robust, but also computationally efficient.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.