Optimization-based statistical methods: functional estimation, sparsity, compound decisions, and deep learning

提出的研究旨在发展和进一步研究基于凸优化的统计方法，同时，其长期目标包括创建一个数学工具箱，用于分析相关优化任务中的扰动，特别是关于有限维和无限维对偶性的相互作用，这是统计理论和计算实现的重要工具。所研究的具体优化问题一般是半无限性质的优化问题：当它们在无限维空间中工作时，它们仍然具有一定的有限维特征。建议的研究领域包括应用于密度估计的凸优化问题的扰动研究，通过Renyi散度估计s-凹密度的申请人工作的延续；将这项研究扩展到其他类似的任务中，并从理论上研究它们在Hadamard意义和鲁棒性方面可能的适定性。这条研究路线的一个建议结果也是用于形状约束密度估计的算法的数值误差界限。基于Renyi散度的估计方法还被进一步提出应用于其他形状约束，如结果估计的单调性，目的是揭示Grenander估计器的一些未知方面，但也在混合模型的背景下，获得具有强加单调性的新型经验贝叶斯预测器。另一个提出的主题是进一步理解在经验贝叶斯框架中产生的混合模型中混合概率的非参数最大似然估计算法；它们随后的多维扩展，切割平面和类似性质，也提出了研究。这一主题延续到经验贝叶斯方法理论的发展，特别是估计贝叶斯预测和“预言”贝叶斯预测的相互作用，所有这些都是在一般损失函数的背景下；本文还提出了后一种情况下某些鲁棒性问题的研究。提出的研究还涉及探索混合模型和深度学习的联系，基于发现某些类型的神经网络，凸神经网络，类似于混合模型；本主题同时旨在研究非参数最大似然估计思想在神经网络中的可能应用，以及相反地，神经网络中某些算法技术在混合模型中概率的非参数估计中的应用。最后提出的领域涉及结构搜索和推理策略的发展，这些策略涉及促进稀疏性的惩罚回归方法，并应用于时间序列和变化点问题的频率分析。

The proposed research aims at the development and further investigation of statistical methodology based on convex optimization, and, at the same time, its long term objectives include the creation of a mathematical toolbox for analyzing perturbations in the pertinent optimization tasks, in particular with regard to an interplay of finite- and infinite-dimensional duality, an important vehicle both for their statistical theory and computational implementations. The specific optimization problems under scrutiny are generally those of semi--infinite character: while they act in infinite-dimensional spaces, they still possess certain finite dimensional features. Proposed areas of investigation comprise the study of perturbations of convex optimization problems applied in density estimation, a continuation of the applicant work in estimation of s-concave densities via Renyi divergences; the extension of this investigation into other similar tasks, and also theoretical investigation of their possible well-posedness in the Hadamard sense and robustness aspects. A proposed outcome of this line of research are also numerical error bounds for the algorithms used in shape-constrained density estimation. The estimation methods based on Renyi divergences are further proposed to be applied also to alternative shape constraints - like the monotonicity of a resulting estimate, with an objective to reveal some yet unknown aspects of the Grenander estimator, but also in the context of mixture models, to obtain novel empirical Bayes predictors with imposed monotonicity. Another proposed topic concerns further understanding of the algorithms for nonparametric maximum likelihood estimation of the mixing probability in mixture models arising in the empirical Bayes framework; their subsequent multidimensional extensions, of cutting-plane and similar nature, are proposed to be investigated as well. This theme continues into the development of the theory of empirical Bayes methods, in particular the interplay of the estimated and "oracle" Bayes predictions, all in the context of general loss functions; the investigation of certain robustness questions in the latter context is proposed as well. The proposed research involves also the exploration of the connection of mixture models and deep learning, based on the discovery that the certain class of neural networks, convex neural networks, are analogous to mixture models; this topic simultaneously aims at the investigation of possible applications of the ideas from nonparametric maximum likelihood estimation to neural networks, and, conversely, that of certain algorithmic techniques from neural networks to the nonparametric estimation of probabilities in mixture models. The last proposed area concerns the development of structure hunting and inference strategies for technologies involving penalized regression methods promoting sparsity, with applications in the frequency analysis of time series and change-point problems.