Collaborative Research: Computational Harmonic Analysis Approach to Active Learning

监督学习的研究关注于揭示训练数据与附加到每个数据的某些函数或标签之间的关系，目标是推广到新的样本。现代机器学习工具，如深度网络，通常需要大量的训练数据，以便以足够的置信度对其余数据进行分类。显然，为基准面指定准确的标签可能是一项昂贵的任务，涉及大量人力。这个项目试图开发出用理论上最少的训练标签对大量数据进行分类的方法。使用少量标签进行分类的关键在于能够选择在哪些数据点查询标签。这个合作研究项目将从几何和调和分析的角度研究这些方法，即主动机器学习，重点是算法洞察力和理论保证。本文将计算调和分析、函数逼近和机器学习中的知识应用到主动学习模型的研究中，重点研究新算法和现有机器学习算法的算法洞察力、高效实现和性能保证。数学工具，包括局域核构造、基于内在维度的逼近分析以及图和流形上算子的本征函数的调和分析，在这些领域的研究中有着天然的应用。具体地说，该项目解决了该领域中出现的四个基本问题：(1)当标签形成具有可能零最小簇间间隔的分层聚类时，如何将采样的标签保守地传播到新的点？(2)核主动学习机制是否推广到图，(3)我们能否将神经网络(或通用参数)分类器的结构合并到所查询的标签的选择中，并可证明地限制了对其余数据的预测的泛化误差？(4)我们如何调整我们的框架以转移学习和高维成像？该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Research in supervised learning is concerned with uncovering relationships between training data and some function or label that is attached to each datum, with the goal of generalizing to new samples. Modern machine learning tools, such as deep networks, typically require a huge set of training data in order to classify the rest of the data with sufficient confidence. Obviously, assigning an accurate label to a datum can be an expensive task, involving a great deal of human effort. This project seeks to develop methods to classify large amounts of data with a theoretically minimal number of training labels. The key to classifying with a small number of labels comes with the ability to choose at which data points a label will be queried. This collaborative research project will study these methods, known as active machine learning, from a geometric and harmonic analysis perspective, focusing on both algorithmic insights and theoretical guarantees. The ability to perform classification with a small number of labeled points has important implications in a variety of applications, including remote sensing classification, medical data analysis, and general applications where it is expensive to collect labels.This project applies knowledge in computational harmonic analysis, function approximation, and machine learning to the study of active learning models, focusing on algorithmic insights, efficient implementations, and performance guarantees for both novel algorithms and currently existing machine learning algorithms. Mathematical tools, including localized kernel construction, approximation analysis in terms of intrinsic dimensionality, and harmonic analysis of eigenfunctions of operators on graphs and manifolds, have natural applications in the study of these areas. Specifically, the project addresses four fundamental questions that arise in the field: (1) How do you conservatively propagate the sampled labels to new points when the labels form a hierarchical clustering with possibly zero minimal separation between clusters? (2) Does the mechanism of kernel active learning generalize to graphs, where naive choice of points to sample becomes a combinatorial optimization problem? (3) Can we incorporate the structure of a neural network (or general parametric) classifier into the choice of labels queried and provably bound the generalization error for predictions on the rest of the data? (4) How can we tailor our framework to transfer learning and high-dimensional imaging?This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.