Collaborative Research: Computational Harmonic Analysis Approach to Active Learning

协作研究：主动学习的计算调和分析方法

• 批准号: 2012266
• 负责人: Alexander Cloninger
• 金额: $ 21.95万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2012266&HistoricalAwards=false
• 关键词: Collaborative; Research; Computational; Harmonic; Analysis

Research in supervised learning is concerned with uncovering relationships between training data and some function or label that is attached to each datum, with the goal of generalizing to new samples. Modern machine learning tools, such as deep networks, typically require a huge set of training data in order to classify the rest of the data with sufficient confidence. Obviously, assigning an accurate label to a datum can be an expensive task, involving a great deal of human effort. This project seeks to develop methods to classify large amounts of data with a theoretically minimal number of training labels. The key to classifying with a small number of labels comes with the ability to choose at which data points a label will be queried. This collaborative research project will study these methods, known as active machine learning, from a geometric and harmonic analysis perspective, focusing on both algorithmic insights and theoretical guarantees. The ability to perform classification with a small number of labeled points has important implications in a variety of applications, including remote sensing classification, medical data analysis, and general applications where it is expensive to collect labels.This project applies knowledge in computational harmonic analysis, function approximation, and machine learning to the study of active learning models, focusing on algorithmic insights, efficient implementations, and performance guarantees for both novel algorithms and currently existing machine learning algorithms. Mathematical tools, including localized kernel construction, approximation analysis in terms of intrinsic dimensionality, and harmonic analysis of eigenfunctions of operators on graphs and manifolds, have natural applications in the study of these areas. Specifically, the project addresses four fundamental questions that arise in the field: (1) How do you conservatively propagate the sampled labels to new points when the labels form a hierarchical clustering with possibly zero minimal separation between clusters? (2) Does the mechanism of kernel active learning generalize to graphs, where naive choice of points to sample becomes a combinatorial optimization problem? (3) Can we incorporate the structure of a neural network (or general parametric) classifier into the choice of labels queried and provably bound the generalization error for predictions on the rest of the data? (4) How can we tailor our framework to transfer learning and high-dimensional imaging?This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

监督学习的研究关注的是发现训练数据与附加到每个数据的某些函数或标签之间的关系，目的是推广到新的样本。现代机器学习工具，如深度网络，通常需要大量的训练数据，才能以足够的置信度对其余数据进行分类。显然，为基准面指定准确的标签可能是一项昂贵的任务，需要大量的人力。该项目旨在开发方法，以理论上最少数量的训练标签对大量数据进行分类。使用少量标签进行分类的关键在于能够选择将在哪些数据点查询标签。这个合作研究项目将从几何和谐波分析的角度研究这些被称为主动机器学习的方法，重点关注算法见解和理论保证。利用少量标记点进行分类的能力在各种应用中具有重要意义，包括遥感分类，医疗数据分析和收集标签昂贵的一般应用。本项目将计算谐波分析，函数逼近和机器学习的知识应用于主动学习模型的研究，专注于算法洞察，高效实现，和现有机器学习算法的性能保证。数学工具，包括本地化核的建设，近似分析的内在维度，和调和分析的本征函数的运营商在图和流形上，有自然的应用在这些领域的研究。具体来说，该项目解决了该领域中出现的四个基本问题：（1）当标签形成聚类之间可能为零的最小分离的分层聚类时，如何保守地将采样标签传播到新点？(2)核主动学习的机制是否可以推广到图中，在图中，对采样点的天真选择变成了一个组合优化问题？(3)我们是否可以将神经网络（或一般参数）分类器的结构结合到查询标签的选择中，并可证明地限制对其余数据的预测的泛化错误？(4)我们如何调整我们的框架来转移学习和高维成像？该奖项反映了NSF的法定使命，并被认为是值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估的支持。

项目成果

StreaMRAK a streaming multi-resolution adaptive kernel algorithm

• DOI: 10.1016/j.amc.2022.127112
• 发表时间: 2021-08
• 期刊: Appl. Math. Comput.
• 作者: Andreas Oslandsbotn;Ž. Kereta;Valeriya Naumova;Y. Freund;A. Cloninger

A Witness Function Based Construction of Discriminative Models Using Hermite Polynomials

• DOI: 10.3389/fams.2020.00031
• 发表时间: 2019-01
• 作者: H. Mhaskar;A. Cloninger;Xiuyuan Cheng

Linear optimal transport embedding: provable Wasserstein classification for certain rigid transformations and perturbations

线性最优传输嵌入：针对某些刚性变换和扰动的可证明 Wasserstein 分类

• DOI: 10.1093/imaiai/iaac023
• 发表时间: 2022
• 期刊: Information and Inference: A Journal of the IMA
• 作者: Moosmüller, Caroline;Cloninger, Alexander
• 通讯作者: Cloninger, Alexander

Nonclosedness of sets of neural networks in Sobolev spaces

• DOI: 10.1016/j.neunet.2021.01.007
• 发表时间: 2021-01
• 期刊: Neural networks : the official journal of the International Neural Network Society
• 作者: Scott Mahan;E. King;A. Cloninger

A Manifold Learning Based Video Prediction Approach for Deep Motion Transfer

一种基于流形学习的深度运动传输视频预测方法

• DOI: 10.1109/iccvw54120.2021.00470
• 发表时间: 2021
• 期刊: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV
• 作者: Cai, Yuliang;Mohan, Sumit;Niranjan, Adithya;Jain, Nilesh;Cloninger, Alexander;Das, Srinjoy
• 通讯作者: Das, Srinjoy