Optimization-based statistical methods: functional estimation, sparsity, compound decisions, and deep learning

点击翻译按钮获取中文摘要

The proposed research aims at the development and further investigation of statistical methodology based on convex optimization, and, at the same time, its long term objectives include the creation of a mathematical toolbox for analyzing perturbations in the pertinent optimization tasks, in particular with regard to an interplay of finite- and infinite-dimensional duality, an important vehicle both for their statistical theory and computational implementations. The specific optimization problems under scrutiny are generally those of semi--infinite character: while they act in infinite-dimensional spaces, they still possess certain finite dimensional features. Proposed areas of investigation comprise the study of perturbations of convex optimization problems applied in density estimation, a continuation of the applicant work in estimation of s-concave densities via Renyi divergences; the extension of this investigation into other similar tasks, and also theoretical investigation of their possible well-posedness in the Hadamard sense and robustness aspects. A proposed outcome of this line of research are also numerical error bounds for the algorithms used in shape-constrained density estimation. The estimation methods based on Renyi divergences are further proposed to be applied also to alternative shape constraints - like the monotonicity of a resulting estimate, with an objective to reveal some yet unknown aspects of the Grenander estimator, but also in the context of mixture models, to obtain novel empirical Bayes predictors with imposed monotonicity. Another proposed topic concerns further understanding of the algorithms for nonparametric maximum likelihood estimation of the mixing probability in mixture models arising in the empirical Bayes framework; their subsequent multidimensional extensions, of cutting-plane and similar nature, are proposed to be investigated as well. This theme continues into the development of the theory of empirical Bayes methods, in particular the interplay of the estimated and "oracle" Bayes predictions, all in the context of general loss functions; the investigation of certain robustness questions in the latter context is proposed as well. The proposed research involves also the exploration of the connection of mixture models and deep learning, based on the discovery that the certain class of neural networks, convex neural networks, are analogous to mixture models; this topic simultaneously aims at the investigation of possible applications of the ideas from nonparametric maximum likelihood estimation to neural networks, and, conversely, that of certain algorithmic techniques from neural networks to the nonparametric estimation of probabilities in mixture models. The last proposed area concerns the development of structure hunting and inference strategies for technologies involving penalized regression methods promoting sparsity, with applications in the frequency analysis of time series and change-point problems.