BIGDATA: F: Random and Adaptive Projections for Scalable Optimization and Learning

海量数据集的结构通常允许它们在几乎没有信息损失的情况下进行压缩。示例包括图像、视频和音频数据，如无处不在的.jpg、。Mpeg和.mp3文件格式。现代数据科学的一个主要目标是利用这种可压缩性，使数据处理算法能够更有效地运行，从而减少丢弃或未分析的数据量。换句话说，数据科学家的目标是调整他们的算法，在不损失性能的情况下直接对压缩数据进行操作。投影是实现压缩的重要计算工具，但到目前为止，它们只被纳入少数数据处理算法中。该项目旨在大幅增加可以从投影中受益的数据处理算法的范围，并特别受到医学图像处理、计算生物学和监控视频分析应用的推动。该项目将产生广泛适用于整个数据科学的算法。此外，这项研究将支持密歇根大学多样化的博士和本科生的跨学科发展，以及密歇根大学机器学习大规模优化研究生课程的开发。该项目的技术目标分为两个重点。第一部分发展了一种将随机投影纳入迭代优化求解器的通用方法，用于解决机器学习中出现的各种优化问题。我们的工作重点是非凸问题，如矩阵分解、流形优化和训练神经网络，建立在项目团队和其他人最近在凸问题上的工作基础上。特别地，我们将扩展迭代草图的原理，这已经发展为凸优化，到非凸问题。该原理找到了将草图（随机投影）插入迭代算法以减少内存或计算复杂性的方法，同时利用数据中的结构来避免性能损失。第二个推力利用第一个推力下开发的算法，开发新的自适应预测，以改善大数据中的统计和计算性能。特别是，这一推力将开发新的方法，用于监督和流主成分分析，以及为必须反复解决的优化问题学习草图，如医学图像形成。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Massive data sets typically have structure that allows them to be compressed with little loss in information. Examples include image, video, and audio data, as reflected by the ubiquitous .jpg, .mpeg, and .mp3 file formats, respectively. A major goal of modern data science is to exploit this compressibility to enable data processing algorithms to run more efficiently, thereby reducing the amount of data that is discarded or left unanalyzed. In other words, data scientists aim to adapt their algorithms to operate directly on the compressed data without loss in performance. Projections are important computational tool for effecting compression, but thus far they have only been incorporated into a handful of data processing algorithms. This project aims to substantially increase the range of data processing algorithms that can benefit from projections, and is specifically motivated by applications to medical image processing, computational biology, and analysis of surveillance video. The project will result in algorithms that are broadly applicable across the whole of data science. Furthermore, this research will support the cross-disciplinary development of a diverse cohort of PhD and undergraduate students at the University of Michigan, and the development of a graduate-level course on large-scale optimization for machine learning at the University of Michigan.The technical aims of the project are divided into two thrusts. The first thrust develops a general approach for incorporating random projections into iterative optimization solvers for broad classes of optimization problems arising in machine learning. Our work focuses on nonconvex problems such as matrix factorization, manifold optimization, and training neural networks, building on recent work by the project team and others on convex problems. In particular, we will extend the principle of iterative sketching, which has been developed for convex optimization, to nonconvex problems. This principle finds ways to insert sketches (random projections) into iterative algorithms so as to reduce memory or computational complexity, while leveraging structures in the data to avoid losses in performance. The second thrust develops new adaptive projections for improved statistical and computational performance in big data, leveraging the algorithms developed under the first thrust. In particular, this thrust will develop new methods for supervised and streaming principle components analysis, and for learning sketches for optimization problems that must be solved repeatedly, as in medical image formation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.