III: Medium: Non-Convex Methods for Discovering High-Dimensional Structures in Big and Corrupted Data

发现高维数据中的结构，如图像、视频和3D点云，已经成为许多学科中科学发现的重要组成部分，包括机器学习、计算机视觉、模式识别和信号处理。这在过去十年中推动了非凡的进步，包括各种基于凸优化的稀疏和低秩建模方法，这些方法具有可证明的正确恢复的理论保证。然而，现有的理论和算法依赖于假设高维数据可以被低维结构很好地近似。虽然这个假设对于某些数据集是足够的，例如，不同光照下的人脸图像，但它可能在许多新兴数据集中被违反，例如，3D点云。该项目的目标是开发一个数学建模框架和相关的非凸优化工具，用于在大数据和损坏数据中发现高维结构。该项目将开发可证明正确且可扩展的优化算法，用于从大数据和损坏数据中学习高维子空间的并集。提出的算法将基于一种称为对偶主成分追踪的新框架，该框架不是为每个子空间学习基，而是寻求学习它们的正交补的基。现有的稀疏和低秩方法要求子空间的维数和离群值的百分比足够小，与之形成鲜明对比的是，所提出的框架将导致即使是最高维数的子空间（即超平面）也可以从高度损坏的数据中正确恢复。这将通过解决一系列非凸稀疏表示问题来实现，这些问题的分析将需要开发新的理论结果来保证从损坏的数据中正确恢复子空间。该项目还将开发可扩展算法来解决这些非凸优化问题，并研究其收敛到全局最优的条件。这些算法将在计算机视觉的两个主要应用中进行评估：点云分割和图像分类数据集的聚类。

Discovering structure in high-dimensional data, such as images, videos and 3D point clouds, has become an essential part of scientific discovery in many disciplines, including machine learning, computer vision, pattern recognition, and signal processing. This has motivated extraordinary advances in the past decade, including various sparse and low-rank modeling methods based on convex optimization with provable theoretical guarantees of correct recovery. However, existing theory and algorithms rely on the assumption that high-dimensional data can be well approximated by low-dimensional structures. While this assumption is adequate for some datasets, e.g., images of faces under varying illumination, it may be violated in many emerging datasets, e.g., 3D point clouds. The goal of this project is to develop a mathematical modeling framework and associated non-convex optimization tools for discovering high-dimensional structures in big and corrupted data.This project will develop provably correct and scalable optimization algorithms for learning a union of high-dimensional subspaces from big and corrupted data. The proposed algorithms will be based on a novel framework called Dual Principal Component Pursuit that, instead of learning a basis for each subspace, seeks to learn a basis for their orthogonal complements. In sharp contrast with existing sparse and low-rank methods, which require both the dimensions of the subspaces and the percentage of outliers to be sufficiently small, the proposed framework will lead to results where even subspaces of the highest possible dimension (i.e., hyperplanes) can be correctly recovered from highly corrupted data. This will be achieved by solving a family of non-convex sparse representation problems whose analysis will require the development of novel theoretical results to guarantee the correct recovery of the subspaces from corrupted data. The project will also develop scalable algorithms for solving these non-convex optimization problems and study conditions for their convergence to the global optimum. These algorithms will be evaluated in two major applications in computer vision: segmentation of point clouds and clustering of image categorization datasets.