Collaborative Research: Computational Harmonic Analysis Approach to Active Learning

点击翻译按钮获取中文摘要

Research in supervised learning is concerned with uncovering relationships between training data and some function or label that is attached to each datum, with the goal of generalizing to new samples. Modern machine learning tools, such as deep networks, typically require a huge set of training data in order to classify the rest of the data with sufficient confidence. Obviously, assigning an accurate label to a datum can be an expensive task, involving a great deal of human effort. This project seeks to develop methods to classify large amounts of data with a theoretically minimal number of training labels. The key to classifying with a small number of labels comes with the ability to choose at which data points a label will be queried. This collaborative research project will study these methods, known as active machine learning, from a geometric and harmonic analysis perspective, focusing on both algorithmic insights and theoretical guarantees. The ability to perform classification with a small number of labeled points has important implications in a variety of applications, including remote sensing classification, medical data analysis, and general applications where it is expensive to collect labels.This project applies knowledge in computational harmonic analysis, function approximation, and machine learning to the study of active learning models, focusing on algorithmic insights, efficient implementations, and performance guarantees for both novel algorithms and currently existing machine learning algorithms. Mathematical tools, including localized kernel construction, approximation analysis in terms of intrinsic dimensionality, and harmonic analysis of eigenfunctions of operators on graphs and manifolds, have natural applications in the study of these areas. Specifically, the project addresses four fundamental questions that arise in the field: (1) How do you conservatively propagate the sampled labels to new points when the labels form a hierarchical clustering with possibly zero minimal separation between clusters? (2) Does the mechanism of kernel active learning generalize to graphs, where naive choice of points to sample becomes a combinatorial optimization problem? (3) Can we incorporate the structure of a neural network (or general parametric) classifier into the choice of labels queried and provably bound the generalization error for predictions on the rest of the data? (4) How can we tailor our framework to transfer learning and high-dimensional imaging?This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.