Complexity Regularization in Statistical Learning Theory

弗拉基米尔Koltchinskii研究了高维统计和机器学习中的两类重要问题：稀疏恢复和流形学习。在这两种情况下，重点是惩罚经验风险最小化凸损失函数和凸复杂性的罚款是用来定义统计估计的目标函数，并在其中的几何性质的问题起着重要的作用的问题。目标之一是扩展谐波分析，信号处理和统计中出现的稀疏恢复理论，超越通常的有限字典框架，以包括机器学习中重要的各种问题（特别是在内核机器方法和集成方法中）。具体而言，其目的是开发一种理论的稀疏恢复的基础上惩罚经验风险最小化的大合奏的内核机器和线性跨度和凸包的无限字典。另一个目标是发展近年来引入的几种流形学习方法的数学理论。这包括与流形相关的偏微分算子的统计估计方法，例如Laplace-Beltrami算子，基于从该流形采样的数据。这些运营商被用来开发一个“近似版本”的调和分析的功能，在非参数函数估计的重要性的流形上。特别是，研究重点是这些运营商的本征值和本征函数的正则化估计的分析，并在流形数据的学习问题的复杂性正则化估计的误差界的发展。该项目是密切相关的数学，统计和计算机科学的几条研究线。更好地理解复杂高维数据集的微妙几何性质，并在开发高维数据的统计推断时考虑到这一点，是统计学和机器学习中非常重要的挑战。稀疏恢复和流形学习是这些领域中最重要的发展之一，其中渐近几何分析，高维概率和微分几何的方法被用来研究一些具有挑战性的统计问题。这导致了新的数学工具和新的统计方法，它们在基于机器学习的方法至关重要的各种领域中具有潜在的应用，例如脑成像，生物信息学，数据和可视化分析。该研究还通过为研究生提供培训机会而使教育受益，并促进了数学，统计学和计算机科学之间的交流与合作。

Vladimir Koltchinskii studies two important classes of problems in High-Dimensional Statistics and Machine Learning: sparse recovery and manifold learning. In both cases, the focus is on the problems in which penalized empirical risk minimization with convex loss functions and convex complexity penalties is used to define statistical estimators of target functions and in which the geometric nature of the problem plays an important role. One of the goals is to extend the theory of sparse recovery that emerged in Harmonic Analysis, Signal Processing and Statistics beyond the usual framework of finite dictionaries to include a variety of problems that are of importance in Machine Learning (in particular, in kernel machines methods and ensemble methods). Specifically, the aim is to develop a theory of sparse recovery based on penalized empirical risk minimization in large ensembles of kernel machines and in linear spans and convex hulls of infinite dictionaries. Another goal is to develop a mathematical theory of several manifold learning methods introduced in the recent years. This includes methods of statistical estimation of partial differential operators associated with a manifold, such as Laplace-Beltrami operator, based on the data sampled from this manifold. These operators are used to develop an ``approximate version'' of harmonic analysis for functions on the manifold that is of importance in nonparametric function estimation. In particular, the research focuses on the analysis of regularized estimators of eigenvalues and eigenfunctions of these operators and on the development of error bounds for complexity regularized estimators in learning problems for manifold data.The project is closely related to several lines of research in Mathematics, Statistics and Computer Science. Better understanding of subtle geometric nature of complex, high-dimensional data sets and taking it into account in the development of statistical inference for high-dimensional data are very important challenges in Statistics and Machine Learning. Sparse recovery and manifold learning are among the most important developments in these areas where the methods of Asymptotic Geometric Analysis, High-Dimensional Probability and Differential Geometry are used to study a number of challenging statistical problems. This leads to new mathematical tools and new statistical methods with potential applications in a variety of areas where the approach based on Machine Learning is crucial, such as Brain Imaging, Bioinformatics, Data and Visual Analytics. The research also benefits education by providing training opportunities for graduate students and it facilitates exchanges and collaborations between Mathematics, Statistics and Computer Science.