CIF: Medium: Collaborative Research: Advances in the Theory and Practice of Low-Rank Matrix Recovery and Modeling

该项目涉及当今科学和工程面临的基本挑战之一，即高效处理和分析大量高维数据，如图像，视频，网页和生物信息学数据。简而言之，数据现在通常存在数千甚至数十亿维度。一方面，大量数据收集的动机是1）科学发现和2）对更好的工程系统的需求。另一方面，目前的困难是在如此高的维度上进行有意义的推理，并从有限的样本数据和有限的计算资源中得出正确的结论。幸运的是，科学或工程数据通常具有非常低的内在复杂性和维度。 该项目解决了这种常见情况所提供的机会，建立了可靠推断实际上是可能的条件，并开发了从庞大数据集中提取关键信息的计算工具。这个跨学科项目预计将产生三个成果：1）研究从部分和损坏的信息中恢复数据矩阵所需的创新数学的发展2）开发有效的算法来恢复低秩矩阵和对损坏的数据进行精确的降维; 3）开发新的应用程序，其中这些技术有望大大提高现有技术的水平。有了这些新的工具，科学家和工程师将能够有效地从数据中提取正确的信息，这是以前无法访问或棘手的传统技术。这将使更好的计算机视觉系统的发展，更好的压缩方案的视频序列，更好地了解基因表达数据，或更好的搜索引擎的网络文件和图像。

This project concerns one of the fundamental challenges facingcontemporary science and engineering today, namely, the efficientprocessing and analysis of massive amounts of high-dimensional data,such as images, videos, web pages, and bioinformatics data. In short,data now routinely lie in thousands or even billions of dimensions. Onthe one hand, massive data collection is motivated by 1) scientificdiscovery and 2) the need for better engineering systems. On the otherhand, the difficult task now is to conduct meaningful inference insuch high dimensions, and draw correct conclusions from limitedamounts of sample data and with limited computationalresources. Fortunately, scientific or engineering data often have verylow intrinsic complexity and dimensionality. This project addressesthe opportunities offered by this common situation, establishesconditions under which reliable inference is actually possible, anddevelops computational tools for extracting key information from hugedata sets.This interdisciplinary project is expected to have three outcomes: 1)the development of innovative mathematics needed to study the recoveryof data matrices from partial and corrupted information 2) thedevelopment of effective algorithms for recovering low-rank matricesand performing accurate dimensionality reduction with corrupted dataand 3) the development of novel applications in which these techniquesare expected to considerably advance the state-of-the-art. With thesenew tools, scientists and engineers will be able to efficientlyextract correct information from data, which was previouslyinaccessible or intractable by conventional techniques. This willenable the development of far better computer vision systems for facerecognition, better compression schemes of video sequences, a betterunderstanding of gene expression data, or better search engines forweb documents and images.