Optimization-based statistical methods: functional estimation, sparsity, compound decisions, and deep learning

基于优化的统计方法：函数估计、稀疏性、复合决策和深度学习

• 批准号: RGPIN-2020-04424
• 负责人: Mizera, Ivan
• 金额: $ 1.31万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=716394
• 关键词: Optimization; based; statistical; methods; functional

The proposed research aims at the development and further investigation of statistical methodology based on convex optimization, and, at the same time, its long term objectives include the creation of a mathematical toolbox for analyzing perturbations in the pertinent optimization tasks, in particular with regard to an interplay of finite- and infinite-dimensional duality, an important vehicle both for their statistical theory and computational implementations. The specific optimization problems under scrutiny are generally those of semi--infinite character: while they act in infinite-dimensional spaces, they still possess certain finite dimensional features. Proposed areas of investigation comprise the study of perturbations of convex optimization problems applied in density estimation, a continuation of the applicant work in estimation of s-concave densities via Renyi divergences; the extension of this investigation into other similar tasks, and also theoretical investigation of their possible well-posedness in the Hadamard sense and robustness aspects. A proposed outcome of this line of research are also numerical error bounds for the algorithms used in shape-constrained density estimation. The estimation methods based on Renyi divergences are further proposed to be applied also to alternative shape constraints - like the monotonicity of a resulting estimate, with an objective to reveal some yet unknown aspects of the Grenander estimator, but also in the context of mixture models, to obtain novel empirical Bayes predictors with imposed monotonicity. Another proposed topic concerns further understanding of the algorithms for nonparametric maximum likelihood estimation of the mixing probability in mixture models arising in the empirical Bayes framework; their subsequent multidimensional extensions, of cutting-plane and similar nature, are proposed to be investigated as well. This theme continues into the development of the theory of empirical Bayes methods, in particular the interplay of the estimated and "oracle" Bayes predictions, all in the context of general loss functions; the investigation of certain robustness questions in the latter context is proposed as well. The proposed research involves also the exploration of the connection of mixture models and deep learning, based on the discovery that the certain class of neural networks, convex neural networks, are analogous to mixture models; this topic simultaneously aims at the investigation of possible applications of the ideas from nonparametric maximum likelihood estimation to neural networks, and, conversely, that of certain algorithmic techniques from neural networks to the nonparametric estimation of probabilities in mixture models. The last proposed area concerns the development of structure hunting and inference strategies for technologies involving penalized regression methods promoting sparsity, with applications in the frequency analysis of time series and change-point problems.

拟议的研究旨在开发和进一步研究基于凸优化的统计方法，同时，其长期目标包括创建一个数学工具箱，用于分析相关优化任务中的扰动，特别是关于有限维和无限维对偶的相互作用，这是统计理论和计算实现的重要工具。具体的最优化问题，在审查一般是那些半无限的字符：虽然他们在无限维空间中的行为，他们仍然具有一定的有限维功能。建议的调查领域包括研究扰动的凸优化问题中应用的密度估计，继续申请人的工作估计的s-凹密度通过Renyi分歧;扩展到其他类似的任务，这一调查，并在理论上调查其可能的适定性在阿达玛意义和鲁棒性方面。这一系列研究的一个建议成果也是形状约束密度估计中使用的算法的数值误差界。基于Renyi分歧的估计方法被进一步提出，也被应用于替代形状约束-如所得到的估计的单调性，其目的是揭示Grenander估计的一些未知方面，但也在混合模型的背景下，以获得具有强加单调性的新的经验贝叶斯预测。另一个建议的主题涉及进一步理解的算法的非参数最大似然估计的混合模型中产生的经验贝叶斯框架中的混合概率，其随后的多维扩展，切割平面和类似的性质，建议进行调查。这一主题继续到经验贝叶斯方法的理论的发展，特别是相互作用的估计和“预言”贝叶斯预测，所有的背景下，一般损失函数;调查的某些鲁棒性问题，在后者的背景下，以及提出。拟议的研究还涉及探索混合模型和深度学习的联系，基于发现某类神经网络，凸神经网络，类似于混合模型;本主题同时旨在研究从非参数最大似然估计到神经网络的思想的可能应用，反之，从神经网络到混合模型中概率的非参数估计的某些算法技术。最后提出的领域涉及发展的结构狩猎和推理策略的技术，涉及惩罚回归方法，促进稀疏性，在时间序列和变点问题的频率分析中的应用。