Probabilistic and Statistical Methods in Machine Learning

机器学习中的概率和统计方法

• 批准号: 0304861
• 负责人: Vladimir Koltchinskii
• 金额: $ 10.08万
• 依托单位: University of New Mexico
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2004
• 资助国家: 美国
• 起止时间: 2004-01-01 至 2006-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0304861&HistoricalAwards=false
• 关键词: Probabilistic; Statistical; Methods; Machine; Learning

The main focus of this research is to take further the theory of data-dependent bounds on generalization error of learning algorithms, especially, in the context of classification problems. One of the main features of modern classification techniques is that they take into account the distribution of the so called classification margins (large margin methods), the quantities that characterize the reliability of classification. However, the margins alone do not give a satisfactory explanation of the superb performance of these methods. To provide such an explanation one has to combine margin type parameters with complexities of the classification rules in rather sophisticated upper confidence bounds. The goal of the research is to use concentration inequalities and various tools from the theory of Gaussian, empirical and Rademacher processes to develop new, much more subtle and powerful bounds of this type, that would lead to a much better understanding of the performance of the existing large margin classification methods and suggest ways to develop statistically optimal large margin procedures. The research includes the study of limit theorems and inequalities for ratio type empirical processes; the study of localized complexities of function classes involved in learning algorithms and the development of new bounds on generalization performance in terms of individual complexities of combined classifiers; the investigation of convergence rates of the empirical margin distribution to the true margin distribution of classifiers in terms of localized and individual complexities; the study of convergence rates of learning algorithms and the development of adaptive classification algorithms with optimal convergence rates; and the investigation of spectral properties of random matrices that play important role in learning theory for kernel machines. Learning Theory is a rapidly growing area between Computer Science, Mathematics, and Statistics that deals with modeling the process of learning and generalization in both biological and artificial neural networks and other learning machines. The results of the research are likely to facilitate further development of boosting, kernel machines, and other learning techniques and lead to new probabilistic bounds and asymptotic results in the theory of empirical processes with potential applications to many problems in statistical learning theory and other areas of Statistics. Methods of statistical learning theory have been penetrating many important areas of applications ranging from biotechnology to computer security. One of the topics of the proposed research is to develop applications of large margin learning methods in the area of robust control, in particular, to the problem of congestion control in communication networks. The results of the research are likely to impact this and other areas of applications.

本研究的主要重点是进一步的学习算法，特别是在分类问题的背景下，推广误差的数据依赖边界的理论。现代分类技术的一个主要特点是，它们考虑到所谓的分类边际（大边际方法）的分布，即表示分类可靠性的数量。然而，仅仅是边际并不能令人满意地解释这些方法的卓越性能。为了提供这样的解释，必须将联合收割机边缘类型参数与相当复杂的置信上限中的分类规则的复杂性相结合。研究的目标是使用浓度不等式和各种工具，从高斯，经验和Rademacher过程的理论，开发新的，更微妙和强大的这种类型的界限，这将导致更好地了解现有的大利润率分类方法的性能，并建议如何开发统计上最优的大利润率程序。研究内容包括比率型经验过程的极限定理和不等式，学习算法中涉及的函数类的局部复杂性的研究，以及组合分类器在个体复杂性方面推广性能的新界的发展;在局部和个体方面，研究分类器的经验边际分布到真实边际分布的收敛速度复杂性;学习算法的收敛速度的研究和具有最佳收敛速度的自适应分类算法的发展;以及在核机器学习理论中发挥重要作用的随机矩阵的谱特性的研究。 学习理论是计算机科学，数学和统计学之间快速发展的领域，涉及生物和人工神经网络以及其他学习机器中的学习和泛化过程建模。研究的结果可能会促进进一步发展的助推，内核机器，和其他学习技术，并导致新的概率界和渐近结果的经验过程理论与潜在的应用，许多问题在统计学习理论和统计的其他领域。统计学习理论的方法已经渗透到许多重要的应用领域，从生物技术到计算机安全。所提出的研究的主题之一是开发大幅度学习方法在鲁棒控制领域的应用，特别是在通信网络中的拥塞控制问题。研究结果可能会影响这一领域和其他应用领域。