Complexity Regularization in Statistical Learning Theory

Vladimir Koltchinskii研究了高维统计和机器学习中的两类重要问题：稀疏恢复和流形学习。在这两种情况下，焦点都集中在具有凸损失函数和凸复杂性惩罚的惩罚经验风险最小化被用来定义目标函数的统计估计的问题，并且问题的几何性质在其中起着重要作用。其中一个目标是将《调和分析、信号处理和统计学》中出现的稀疏恢复理论扩展到有限词典的通常框架之外，以包括机器学习中的各种重要问题(特别是在核机器方法和集成方法中)。具体地说，目标是在核机器的大型集成以及无限字典的线性跨度和凸壳中发展基于惩罚经验风险最小化的稀疏恢复理论。另一个目标是发展近年来引入的几种流形学习方法的数学理论。这包括基于从该流形采样的数据对与流形相关联的偏微分算子的统计估计方法，例如Laplace-Beltrami算子。这些算子被用来为流形上的函数发展一种“近似版本”的调和分析，这在非参数函数估计中是重要的。特别地，该研究集中于对这些算子的特征值和特征函数的正则化估计的分析，以及流形数据学习问题中复杂性正则化估计的误差界的发展。更好地理解复杂高维数据集的精细几何性质，并在高维数据的统计推理中考虑到这一点，是统计学和机器学习中非常重要的挑战。稀疏恢复和流形学习是这些领域最重要的发展之一，在这些领域中，渐近几何分析、高维概率和微分几何方法被用于研究一些具有挑战性的统计问题。这导致了新的数学工具和新的统计方法，在基于机器学习的方法至关重要的各种领域中具有潜在的应用，如脑成像、生物信息学、数据和视觉分析。这项研究也有利于教育，为研究生提供培训机会，促进数学、统计学和计算机科学之间的交流与合作。

Vladimir Koltchinskii studies two important classes of problems in High-Dimensional Statistics and Machine Learning: sparse recovery and manifold learning. In both cases, the focus is on the problems in which penalized empirical risk minimization with convex loss functions and convex complexity penalties is used to define statistical estimators of target functions and in which the geometric nature of the problem plays an important role. One of the goals is to extend the theory of sparse recovery that emerged in Harmonic Analysis, Signal Processing and Statistics beyond the usual framework of finite dictionaries to include a variety of problems that are of importance in Machine Learning (in particular, in kernel machines methods and ensemble methods). Specifically, the aim is to develop a theory of sparse recovery based on penalized empirical risk minimization in large ensembles of kernel machines and in linear spans and convex hulls of infinite dictionaries. Another goal is to develop a mathematical theory of several manifold learning methods introduced in the recent years. This includes methods of statistical estimation of partial differential operators associated with a manifold, such as Laplace-Beltrami operator, based on the data sampled from this manifold. These operators are used to develop an ``approximate version'' of harmonic analysis for functions on the manifold that is of importance in nonparametric function estimation. In particular, the research focuses on the analysis of regularized estimators of eigenvalues and eigenfunctions of these operators and on the development of error bounds for complexity regularized estimators in learning problems for manifold data.The project is closely related to several lines of research in Mathematics, Statistics and Computer Science. Better understanding of subtle geometric nature of complex, high-dimensional data sets and taking it into account in the development of statistical inference for high-dimensional data are very important challenges in Statistics and Machine Learning. Sparse recovery and manifold learning are among the most important developments in these areas where the methods of Asymptotic Geometric Analysis, High-Dimensional Probability and Differential Geometry are used to study a number of challenging statistical problems. This leads to new mathematical tools and new statistical methods with potential applications in a variety of areas where the approach based on Machine Learning is crucial, such as Brain Imaging, Bioinformatics, Data and Visual Analytics. The research also benefits education by providing training opportunities for graduate students and it facilitates exchanges and collaborations between Mathematics, Statistics and Computer Science.