AF: Small: Data Stream Algorithms with Application to Linear Algebra

AF：小：数据流算法及其在线性代数中的应用

• 批准号: 1815840
• 负责人: David Woodruff
• 金额: $ 43.4万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-06-01 至 2023-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815840&HistoricalAwards=false
• 关键词: AF; Small; Data; Stream; Algorithms

Many important problems in machine learning, scientific computing, and statistics benefit from fast procedures for solving numerical linear algebra problems. At the same time, many large-scale datasets, such as internet search logs, network traffic, and sensor network data, have been studied in data-stream literature. Surprisingly, a number of fast procedures used in numerical linear algebra have been made possible by exploiting tools that were originally developed in the data-stream literature. These developments are based on a technique called sketching, which is a tool for quickly compressing a problem to a smaller version of itself, for which one can then afford to run a much slower procedure on the smaller problem. A major goal of this project is to study foundational problems in the data-stream domain, and develop their connections to problems in numerical linear algebra. The algorithms developed here will be accessible to graduate and undergraduate students from computer science, machine learning, and mathematics, and the investigator plans to integrate the results of the project into a graduate course on algorithms for big data, as well as undergraduate algorithms courses. The investigator is actively working with underrepresented minority and undergraduate researchers on topics directly related to this project.The first main thrust of this project is to develop new techniques for fundamental problems in data streams where the goal is to use minimal amount of memory while running algorithms over one pass on a data stream. Such problems include statistical problems - such as estimating the variance, moments, most frequent items, heavy hitters - for many of which optimal memory bounds are still unknown. Another challenge in this domain is that of processing massive streams for real-world graphs, such as geometric intersection graphs, which arise in cellular networks and scheduling theory. By studying a breadth of problems in different corners of data streams, the investigator plans to develop new techniques and build unexpected applications. The second main thrust of the project is to develop new algorithms and hardness results for fundamental problems in numerical linear algebra, connecting such problems to the study of data streams. CountSketch, which is a low-memory heavy hitters algorithm, paved the way for obtaining time-optimal algorithms for regression. This project will continue to push forward such connections, for example, by studying CountSketch in the context of tensors, which is a new application domain. The project will also investigate many deterministic (instead of randomized) data stream algorithms, and understanding their role in linear algebra. A major goal is to understand the limitations of speedups to linear algebra problems. One particular problem of interest here is to obtain low rank approximation with spectral norms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习、科学计算和统计学中的许多重要问题都得益于快速求解数值线性代数问题的过程。同时，在数据流文献中，研究了许多大规模的数据集，如互联网搜索日志、网络流量和传感器网络数据。令人惊讶的是，通过利用最初在数据流文献中开发的工具，许多用于数值线性代数的快速过程已经成为可能。这些发展是基于一种称为草图的技术，它是一种将问题快速压缩为自身较小版本的工具，因此可以在较小的问题上运行慢得多的过程。该项目的主要目标是研究数据流领域的基础问题，并发展它们与数值线性代数问题的联系。这里开发的算法将对计算机科学、机器学习和数学专业的研究生和本科生开放，研究者计划将该项目的结果整合到大数据算法的研究生课程以及本科生算法课程中。研究者积极地与未被充分代表的少数民族和本科生研究人员就与本项目直接相关的主题进行合作。该项目的第一个主要目标是为数据流中的基本问题开发新技术，其目标是在数据流上运行算法的一次传递时使用最少的内存。这些问题包括统计问题——例如估计方差、时刻、最频繁的项目、重磅打击——其中许多问题的最佳记忆界限仍然未知。该领域的另一个挑战是处理现实世界图的大量流，例如蜂窝网络和调度理论中出现的几何相交图。通过研究数据流不同角落的广泛问题，研究者计划开发新技术并构建意想不到的应用。该项目的第二个主要目标是为数值线性代数的基本问题开发新的算法和硬度结果，将这些问题与数据流的研究联系起来。countssketch是一个低内存的重量级算法，它为获得时间最优的回归算法铺平了道路。该项目将继续推进这种联系，例如，通过研究张量背景下的countssketch，这是一个新的应用领域。该项目还将研究许多确定性（而不是随机）数据流算法，并了解它们在线性代数中的作用。一个主要的目标是理解加速对线性代数问题的限制。这里一个特别有趣的问题是用谱范数获得低秩近似。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Subspace exploration: Bounds on Projected Frequency Estimation.

• DOI: 10.1145/3452021.3458312
• 发表时间: 2021-06
• 期刊: Proceedings of the ... ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems
• 作者: Cormode G;Dickens C;Woodruff DP
• 通讯作者: Woodruff DP

Span Recovery for Deep Neural Networks with Applications to Input Obfuscation

深度神经网络的跨度恢复及其输入混淆的应用

• 发表时间: 2020
• 期刊: ICLR
• 作者: Jayaram, Rajesh;Woodruff, David P.;Zhang, Richard
• 通讯作者: Zhang, Richard

Matrix Sketching with Applications to Regression and Optimization

矩阵草图及其在回归和优化中的应用

• 发表时间: 2021
• 期刊: Proceedings of Machine Learning Research
• 作者: Feng, Zhili;Roosta-Khorasani, Fred;Woodruff, David P.
• 通讯作者: Woodruff, David P.

Perfect $L_p$ Sampling in a Data Stream

数据流中的完美 $L_p$ 采样

• DOI: 10.1137/18m1229912
• 发表时间: 2021
• 期刊: SIAM Journal on Computing
• 影响因子: 1.6
• 作者: Jayaram, Rajesh;Woodruff, David
• 通讯作者: Woodruff, David

The ℓp-Subspace Sketch Problem in Small Dimensions with Applications to Support Vector Machines

小维中的 β 子空间草图问题及其支持向量机的应用

• 发表时间: 2023
• 期刊: SODA
• 作者: Li, Yi;Lin, Honghao;Woodruff, David P.
• 通讯作者: Woodruff, David P.