Optimization-based statistical methods: functional estimation, sparsity, compound decisions, and deep learning

拟议的研究旨在开发和进一步研究基于凸优化的统计方法，同时，其长期目标包括创建一个数学工具箱，用于分析有关优化任务中的扰动，特别是关于有限和无限维对偶的相互作用，这是其统计理论和计算实施的重要工具。所研究的具体优化问题一般都是半无限性质的：当它们作用于无限维空间时，它们仍然具有一定的有限维特征。提出的研究领域包括应用于密度估计的凸优化问题的摄动研究，S-凹密度估计的应用工作的继续，这一研究的扩展到其他类似的任务，以及它们在Hadamard意义和稳健性方面的可能适定性的理论研究。这一系列研究的一个拟议结果也是用于形状约束密度估计的算法的数值误差界。基于Renyi发散的估计方法被进一步建议应用于另一种形状约束，如结果估计的单调性，目的是揭示Griander估计的一些未知方面，但也在混合模型的背景下，获得具有强制单调性的新的经验贝叶斯预报器。另一个提出的主题涉及对经验贝叶斯框架中混合模型混合概率的非参数极大似然估计算法的进一步理解，以及它们随后的割平面和类似性质的多维扩展。这一主题继续发展经验贝叶斯方法的理论，特别是估计贝叶斯预测和“先知”贝叶斯预测的相互作用，所有这些都是在一般损失函数的背景下进行的；还提出了在后一种背景下对某些稳健性问题的研究。基于某些神经网络，即凸神经网络，类似于混合模型，本研究还涉及混合模型与深度学习之间的联系的探索；本课题同时旨在研究非参数极大似然估计思想在神经网络中的可能应用，反之，从神经网络到混合模型中概率的非参数估计的某些算法技术的可能应用。最后提出的领域涉及结构搜索和推理策略的发展，涉及促进稀疏性的惩罚回归方法的技术，以及在时间序列和变点问题的频率分析中的应用。

The proposed research aims at the development and further investigation of statistical methodology based on convex optimization, and, at the same time, its long term objectives include the creation of a mathematical toolbox for analyzing perturbations in the pertinent optimization tasks, in particular with regard to an interplay of finite- and infinite-dimensional duality, an important vehicle both for their statistical theory and computational implementations. The specific optimization problems under scrutiny are generally those of semi--infinite character: while they act in infinite-dimensional spaces, they still possess certain finite dimensional features. Proposed areas of investigation comprise the study of perturbations of convex optimization problems applied in density estimation, a continuation of the applicant work in estimation of s-concave densities via Renyi divergences; the extension of this investigation into other similar tasks, and also theoretical investigation of their possible well-posedness in the Hadamard sense and robustness aspects. A proposed outcome of this line of research are also numerical error bounds for the algorithms used in shape-constrained density estimation. The estimation methods based on Renyi divergences are further proposed to be applied also to alternative shape constraints - like the monotonicity of a resulting estimate, with an objective to reveal some yet unknown aspects of the Grenander estimator, but also in the context of mixture models, to obtain novel empirical Bayes predictors with imposed monotonicity. Another proposed topic concerns further understanding of the algorithms for nonparametric maximum likelihood estimation of the mixing probability in mixture models arising in the empirical Bayes framework; their subsequent multidimensional extensions, of cutting-plane and similar nature, are proposed to be investigated as well. This theme continues into the development of the theory of empirical Bayes methods, in particular the interplay of the estimated and "oracle" Bayes predictions, all in the context of general loss functions; the investigation of certain robustness questions in the latter context is proposed as well. The proposed research involves also the exploration of the connection of mixture models and deep learning, based on the discovery that the certain class of neural networks, convex neural networks, are analogous to mixture models; this topic simultaneously aims at the investigation of possible applications of the ideas from nonparametric maximum likelihood estimation to neural networks, and, conversely, that of certain algorithmic techniques from neural networks to the nonparametric estimation of probabilities in mixture models. The last proposed area concerns the development of structure hunting and inference strategies for technologies involving penalized regression methods promoting sparsity, with applications in the frequency analysis of time series and change-point problems.