Non-Parametric Estimation under Shape/Norm Constraints

点击翻译按钮获取中文摘要

This project is about advancing methodology and theoretical understanding of certain algorithms that are widely used today in machine learning and statistics. Decision Trees are an important type of predictive modelling method. They have a long history and modern variations like random forest are among the most powerful techniques available. One part of the project would develop new and theoretically sound decision trees accompanied by software implementation. The other part of the project applies to signal processing. Total Variation Denoising is a popular method used in image processing to do noise removal. The current algorithm as it stands, is not fully automated. This project would develop a fully automated version of this algorithm which is theoretically valid. In the bigger picture, the research would generate improved versions of these time tested algorithms and our understanding of how and why they work would be refined. A main focus of the project is to study non parametric estimation of non smooth functions such as piecewise constant/linear/polynomial functions, in high dimensions. One major agenda here is to give theoretical guarantees for CART like estimators. These guarantees would provably demonstrate adaptivity to the number and arrangement of rectangular level sets of the regression function. Theoretical understanding of such adaptivity for CART like estimators are largely absent in the literature and this research should be a first step towards filling this gap. In this project, an extension of the Dyadic CART estimator, called Model Selection Cart (MS Cart) is proposed as a computationally and theoretically tractable method to achieve the desired adaptivity. We have already developed an algorithm (to be implemented and made publicly available), based on a dynamic programming approach, which provably computes the MS Cart estimator efficiently. We are currently working on showing theoretical guarantees for MS Cart. Another main focus of the proposal is to study the methodology of Total Variation Denoising (TVD). This technique is a non linear image denoising technique heavily used in the image processing community. The first problem talked about in this proposal, under this topic, is a step towards rigorous understanding of the statistical risk of the TVD estimator. Worst case analysis of this risk is now well understood in the literature. The research proposed here will go beyond worst case analysis and reveal the adaptivity of the TVD estimator. The second proposed problem deals with the very practical issue of choosing the tuning parameter for TVD in a fully data driven way. A new tuning parameter free estimator is proposed here whose practical performance has been thoroughly checked by us in simulations. We will show that our proposed estimator is minimax rate optimal while being fully data driven. Such an estimator is not available in the literature, as of now. This estimator has the potential to make the TVD methodology much more user friendly than it already is as choosing a tuning parameter can be a delicate issue. The PI will also address several non parametric estimation problems in settings such as quantile regression and in general exponential families. In particular, one agenda here is to study shape constrained estimation in quantile regression which would be a first step towards going beyond restrictive linear assumptions often made in the existing literature.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.