Task-Aware Quantization in Data Science: Theory and Fast Algorithms

机器学习算法无处不在，它们在数据科学中的应用正在增加。该项目的重点是在数据科学应用中开发计算效率高的算法，其中离散化（也称为量化）起着重要作用。在这里，量化是用有限集合中的元素替换真实的数的过程，比如从传感器测量中获得的那些。这使得它们能够有效地进行数字表示、存储、压缩和传输。感兴趣的应用包括深度学习，这一领域在一系列令人惊叹的领域取得了轰动性的突破。其前沿领域之一是在硬件上构建神经网络，这些硬件可以放入手持和可穿戴设备以及智能家居中。为此，神经网络必须被有效地量化;该项目的一个关键目标是为这项任务设计算法。另一个应用涉及边缘设备，例如传感器网络中的传感器，它们在严格的功率限制下进行通信和执行计算。该项目的目标是开发计算效率高的算法，用于量化和压缩数据，以减少功耗。第三个应用涉及推荐系统，它收集用户对产品的离散化评级，并将其转换为其他产品推荐。该项目通过参与研究为研究生提供培训。该项目侧重于开发具有可证明误差保证的计算效率高的量化算法。它由三个重要的应用领域驱动。首先，在目标是离散化函数参数的设置中，例如在深度神经网络的压缩中，它寻求量化算法来生成功能等效的网络，这些网络需要存储更少的比特。第二个激励领域涉及到推理任务必须在边缘设备上完成的设置，在通信和计算约束下，就像在传感器网络中一样。在这里，重点是计算效率的测量，量化和推理算法，需要最小的内存和功耗要求。第三，在应用中，信号恢复的目标和测量是固有的二进制和昂贵的收集，如在推荐系统中，重点是设计和研究有效的自适应算法的顺序选择的测量。该项目旨在开发最先进的任务感知算法，需要开发和使用数学多个领域的工具，包括几何泛函分析和非渐近随机矩阵理论的方法。与框架理论，压缩感知和噪声整形量化的连接也将建立。在分析的算法，离散几何，优化和数值分析技术将被开发和使用。为了将与该项目相关的理论保证与最佳保证进行比较，近似理论将是必不可少的。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning algorithms are ubiquitous, and their applications in data science are on the rise. This project focuses on developing computationally efficient algorithms in data-science applications where discretization, also known as quantization, plays a fundamental role. Here quantization is the process that replaces real numbers, like those obtained from sensor measurements, by elements in a finite set. This makes them amenable to efficient digital representation, storage, compression, and transmission. Applications of interest include deep learning, an area that has led to sensational breakthroughs in a stunning range of areas. One of its frontiers is building neural networks on hardware that can be put into handheld and wearable devices as well as those in smart homes. For that, neural networks must be efficiently quantized; a key goal of this project is to devise algorithms for this task. Another application concerns edge devices, such as sensors in a sensor network, which communicate and perform computations under severe power limitations. A goal of this project is to develop computationally efficient algorithms for quantizing and compressing their data to enable reducing power use. A third application involves recommender systems, which collect users’ discretized ratings of products and transform them into other product recommendations for others. The project provides training for graduate students through involvement in the research.This project focuses on developing computationally efficient quantization algorithms with provable error guarantees. It is motivated by three important application areas. First, in settings where the goal is discretizing the parameters of a function, as in the compression of deep neural networks, it seeks quantization algorithms to generate functionally equivalent networks that require many fewer bits to store. The second motivating area involves settings where inference tasks must be done on edge-devices, under communication and computation constraints, as in sensor-networks. Here, the focus is on computationally efficient measurement, quantization, and inference algorithms that entail minimal memory and power requirements. Third, in applications where signal recovery is the goal and measurements are inherently binary and expensive to collect, as in recommender systems, the focus is on devising and studying efficient adaptive algorithms for sequential selection of the measurements. This project, which aims to develop state of the art task-aware algorithms, entails developing and using tools from several areas of mathematics, including methods from geometric functional analysis and non-asymptotic random matrix theory. Connections with frame theory, compressed sensing, and noise-shaping quantization will also be established. In analyzing the algorithms, discrete geometry, optimization, and numerical analysis techniques will be developed and employed. To compare theoretical guarantees associated with this project with best possible ones, approximation theory will be essential.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.