CAREER: Robust Algorithms for Corrupted Data

近年来，数据驱动的科学发现和技术创新方法有所增加。巨大规模和种类的数据集通常用于探索新现象和训练算法。然而，随着数据量和复杂性的增长，数据质量往往会下降。数据可能受到噪声、异常值、缺失值和其他形式的信息丢失的困扰。该项目将设计健壮、高效的算法来分析这些损坏的数据。这些方法将用于成像、生物学和其他高影响领域的问题。它们将以通用代码的形式实现，并发布给公众使用。除了研究产品外，该项目还将创造教育资源，培养本科生和研究生初级阶段的学生在数据分析的理论和实践方面的能力。将考虑两种互补的方法来处理损坏的数据。对于第一类方法，将假设数据具有潜在的低秩结构，该结构受到加性噪声和线性滤波器的扰动。随机矩阵理论的新结果将用于信号恢复问题，包括协方差和距离矩阵的估计。第二类方法将利用数据中的几何结构。计算调和分析的新方法将被设计出来，通过揭示数据点之间的关系信息来学习数据集的几何结构，并将这些几何信息用于聚类、回归和其他任务。这两种方法都有可能产生不仅在统计上稳健，而且在计算上高效的方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent years have seen an increase in data-driven approaches to scientific discovery and technological innovation. Datasets of enormous size and variety are routinely used to explore new phenomena and train algorithms. As data volume and complexity grow, however, data quality often decreases. Data can be plagued by noise, outliers, missing values, and other forms of information loss. This project will design robust, efficient algorithms for analyzing such corrupted data. These methods will be deployed on problems in imaging, biology, and other high-impact domains. They will be implemented in general-purpose codes, to be released for public use. In addition to its research products, this project will also create educational resources to train students at the advanced undergraduate and beginning graduate level in the theory and practice of data analysis.Two complementary methodologies will be considered for processing corrupted data. For the first class of methods, the data will be assumed to have an underlying low-rank structure that is perturbed by both additive noise and linear filters. New results from random matrix theory will be developed for signal recovery problems in this setting, including the estimation of covariance and distance matrices. The second class of methods will exploit geometric structures in data. Novel methods from computational harmonic analysis will be devised to both learn the geometry of a dataset by uncovering relational information between data points, and to use this geometric information for clustering, regression, and other tasks. Both approaches have the potential to yield methods that are not only statistically robust, but also computationally efficient.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.