RI: Small: Non-parametric Machine Learning in the Age of Deep and High-Dimensional Models

RI：小：深度和高维模型时代的非参数机器学习

• 批准号: 1909816
• 负责人: Pradeep Ravikumar
• 金额: $ 44.99万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909816&HistoricalAwards=false
• 关键词: RI; Small; Non; parametric; Machine

The empirical successes of machine learning in recent years are due, in a large part, to models that are highly flexible (such as deep neural networks), which can tackle complex tasks (such as predicting whether a medical image is indicative of cancer). These models learn from large datasets and require a lot of empirical engineering expertise. The current belief is that a better understanding of this engineering practice might help us gain insights into the relationship between the models and the sample size of the datasets. Taking inspirations for deep neural networks, the research in this project formulates broader classes of models and algorithms that learn from fewer samples but do not require complex engineering expertise. The research enables learning highly flexible models in a more rigorous and reproducible manner; consequently, the users may have greater trust in the resulting applications.More specifically, the research in this project leverages insights from deep and high-dimensional models to develop a new class of non-parametric prediction as well as density functions. The project develops novel extensions of parametric structural sparsity constraints to the non-parametric estimation setting. By treating the multivariate prediction functions as functional generalizations of tensors, the project develops novel extensions of structural sparsity constraints designed for parametric model parameters to novel counterparts for prediction functions. The project also investigates a "destructive learning" approach to learning deep compositional models, which have a similar compositional form to deep neural network models. The project develops stage-wise algorithms to learn such deep compositional models, similar to boosting, by iteratively finding and destroying information in the data using well-studied shallow learning algorithms in each stage.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，机器学习在经验上取得了成功，这在很大程度上要归功于高度灵活的模型(如深度神经网络)，这些模型可以处理复杂的任务(如预测医学图像是否预示着癌症)。这些模型从大数据集中学习，需要大量的经验工程专业知识。目前的看法是，更好地理解这一工程实践可能有助于我们深入了解模型与数据集样本大小之间的关系。受深度神经网络的启发，该项目的研究制定了更广泛的模型和算法类别，这些模型和算法从较少的样本中学习，但不需要复杂的工程专业知识。该研究能够以更严格和可重复性的方式学习高度灵活的模型；因此，用户可能对所产生的应用程序更有信心。更具体地说，该项目的研究利用来自深层次和高维模型的见解来开发一类新的非参数预测以及密度函数。该项目将参数结构稀疏约束的新扩展扩展到非参数估计设置。通过将多变量预测函数视为张量的函数推广，该项目开发了针对参数模型参数设计的结构稀疏约束的新扩展到针对预测函数的新对应约束。该项目还研究了一种学习深层组成模型的“破坏性学习”方法，该模型具有类似于深层神经网络模型的组成形式。该项目开发了阶段性算法，通过在每个阶段使用经过充分研究的浅层学习算法迭代地寻找和销毁数据中的信息，来学习这样的深层组成模型，类似于增强。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

IDENTIFIABILITY OF NONPARAMETRIC MIXTURE MODELS AND BAYES OPTIMAL CLUSTERING

• DOI: 10.1214/19-aos1887
• 发表时间: 2020-08-01
• 期刊: ANNALS OF STATISTICS
• 影响因子: 4.5
• 作者: Aragam, Bryon;Dan, Chen;Ravikumar, Pradeep
• 通讯作者: Ravikumar, Pradeep

Boosted CVaR Classification

• 发表时间: 2021-10
• 期刊: ArXiv
• 作者: Runtian Zhai;Chen Dan;A. Suggala;Zico Kolter;Pradeep Ravikumar

Identifiability of deep generative models without auxiliary information

• 发表时间: 2022-06
• 作者: Bohdan Kivva;Goutham Rajendran;Pradeep Ravikumar;Bryon Aragam

Learning Sparse Nonparametric DAGs

• 发表时间: 2019-09
• 期刊: ArXiv
• 作者: Xun Zheng;Chen Dan;Bryon Aragam;Pradeep Ravikumar;E. Xing

Learning Linear Causal Representations from Interventions under General Nonlinear Mixing

• DOI: 10.48550/arxiv.2306.02235
• 发表时间: 2023-06
• 期刊: ArXiv
• 作者: Simon Buchholz;Goutham Rajendran;Elan Rosenfeld;Bryon Aragam;B. Scholkopf;Pradeep Ravikumar