RI: Small: Non-parametric Machine Learning in the Age of Deep and High-Dimensional Models

点击翻译按钮获取中文摘要

The empirical successes of machine learning in recent years are due, in a large part, to models that are highly flexible (such as deep neural networks), which can tackle complex tasks (such as predicting whether a medical image is indicative of cancer). These models learn from large datasets and require a lot of empirical engineering expertise. The current belief is that a better understanding of this engineering practice might help us gain insights into the relationship between the models and the sample size of the datasets. Taking inspirations for deep neural networks, the research in this project formulates broader classes of models and algorithms that learn from fewer samples but do not require complex engineering expertise. The research enables learning highly flexible models in a more rigorous and reproducible manner; consequently, the users may have greater trust in the resulting applications.More specifically, the research in this project leverages insights from deep and high-dimensional models to develop a new class of non-parametric prediction as well as density functions. The project develops novel extensions of parametric structural sparsity constraints to the non-parametric estimation setting. By treating the multivariate prediction functions as functional generalizations of tensors, the project develops novel extensions of structural sparsity constraints designed for parametric model parameters to novel counterparts for prediction functions. The project also investigates a "destructive learning" approach to learning deep compositional models, which have a similar compositional form to deep neural network models. The project develops stage-wise algorithms to learn such deep compositional models, similar to boosting, by iteratively finding and destroying information in the data using well-studied shallow learning algorithms in each stage.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.