Probabilistic and Statistical Methods in Machine Learning

本研究的主要重点是进一步研究学习算法泛化误差的数据依赖边界理论，特别是在分类问题的背景下。现代分类技术的一个主要特点是，它们考虑到所谓的分类边际（大边际方法）的分布，即分类可靠性的数量特征。然而，仅靠边缘并不能令人满意地解释这些方法的卓越性能。为了提供这样的解释，必须将边际类型参数与相当复杂的上置信区间的分类规则的复杂性结合起来。本研究的目标是利用集中不等式和各种工具，从高斯理论、经验和Rademacher过程中开发新的、更微妙和强大的这种类型的边界，这将导致更好地理解现有的大边际分类方法的性能，并提出开发统计上最优的大边际程序的方法。主要研究了比率型经验过程的极限定理和不等式；学习算法中涉及的函数类的局部复杂性的研究，以及根据组合分类器的单个复杂性对泛化性能的新界限的发展；局部复杂性和个体复杂性下分类器的经验边际分布对真实边际分布的收敛率研究学习算法收敛率的研究和最优收敛率的自适应分类算法的发展随机矩阵谱性质的研究在核机器的学习理论中起着重要的作用。学习理论是介于计算机科学、数学和统计学之间的一个快速发展的领域，它涉及在生物和人工神经网络以及其他学习机器中学习和泛化过程的建模。研究结果可能会促进推进、核机和其他学习技术的进一步发展，并在经验过程理论中产生新的概率界和渐近结果，并可能应用于统计学习理论和其他统计领域的许多问题。统计学习理论的方法已经渗透到许多重要的应用领域，从生物技术到计算机安全。提出的研究课题之一是开发大边际学习方法在鲁棒控制领域的应用，特别是在通信网络中的拥塞控制问题上。这项研究的结果可能会影响这一领域和其他领域的应用。

The main focus of this research is to take further the theory of data-dependent bounds on generalization error of learning algorithms, especially, in the context of classification problems. One of the main features of modern classification techniques is that they take into account the distribution of the so called classification margins (large margin methods), the quantities that characterize the reliability of classification. However, the margins alone do not give a satisfactory explanation of the superb performance of these methods. To provide such an explanation one has to combine margin type parameters with complexities of the classification rules in rather sophisticated upper confidence bounds. The goal of the research is to use concentration inequalities and various tools from the theory of Gaussian, empirical and Rademacher processes to develop new, much more subtle and powerful bounds of this type, that would lead to a much better understanding of the performance of the existing large margin classification methods and suggest ways to develop statistically optimal large margin procedures. The research includes the study of limit theorems and inequalities for ratio type empirical processes; the study of localized complexities of function classes involved in learning algorithms and the development of new bounds on generalization performance in terms of individual complexities of combined classifiers; the investigation of convergence rates of the empirical margin distribution to the true margin distribution of classifiers in terms of localized and individual complexities; the study of convergence rates of learning algorithms and the development of adaptive classification algorithms with optimal convergence rates; and the investigation of spectral properties of random matrices that play important role in learning theory for kernel machines. Learning Theory is a rapidly growing area between Computer Science, Mathematics, and Statistics that deals with modeling the process of learning and generalization in both biological and artificial neural networks and other learning machines. The results of the research are likely to facilitate further development of boosting, kernel machines, and other learning techniques and lead to new probabilistic bounds and asymptotic results in the theory of empirical processes with potential applications to many problems in statistical learning theory and other areas of Statistics. Methods of statistical learning theory have been penetrating many important areas of applications ranging from biotechnology to computer security. One of the topics of the proposed research is to develop applications of large margin learning methods in the area of robust control, in particular, to the problem of congestion control in communication networks. The results of the research are likely to impact this and other areas of applications.