Task-Aware Quantization in Data Science: Theory and Fast Algorithms

点击翻译按钮获取中文摘要

Machine learning algorithms are ubiquitous, and their applications in data science are on the rise. This project focuses on developing computationally efficient algorithms in data-science applications where discretization, also known as quantization, plays a fundamental role. Here quantization is the process that replaces real numbers, like those obtained from sensor measurements, by elements in a finite set. This makes them amenable to efficient digital representation, storage, compression, and transmission. Applications of interest include deep learning, an area that has led to sensational breakthroughs in a stunning range of areas. One of its frontiers is building neural networks on hardware that can be put into handheld and wearable devices as well as those in smart homes. For that, neural networks must be efficiently quantized; a key goal of this project is to devise algorithms for this task. Another application concerns edge devices, such as sensors in a sensor network, which communicate and perform computations under severe power limitations. A goal of this project is to develop computationally efficient algorithms for quantizing and compressing their data to enable reducing power use. A third application involves recommender systems, which collect users’ discretized ratings of products and transform them into other product recommendations for others. The project provides training for graduate students through involvement in the research.This project focuses on developing computationally efficient quantization algorithms with provable error guarantees. It is motivated by three important application areas. First, in settings where the goal is discretizing the parameters of a function, as in the compression of deep neural networks, it seeks quantization algorithms to generate functionally equivalent networks that require many fewer bits to store. The second motivating area involves settings where inference tasks must be done on edge-devices, under communication and computation constraints, as in sensor-networks. Here, the focus is on computationally efficient measurement, quantization, and inference algorithms that entail minimal memory and power requirements. Third, in applications where signal recovery is the goal and measurements are inherently binary and expensive to collect, as in recommender systems, the focus is on devising and studying efficient adaptive algorithms for sequential selection of the measurements. This project, which aims to develop state of the art task-aware algorithms, entails developing and using tools from several areas of mathematics, including methods from geometric functional analysis and non-asymptotic random matrix theory. Connections with frame theory, compressed sensing, and noise-shaping quantization will also be established. In analyzing the algorithms, discrete geometry, optimization, and numerical analysis techniques will be developed and employed. To compare theoretical guarantees associated with this project with best possible ones, approximation theory will be essential.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.