III: Small: Collaborative Research: Probabilistic Models using Generalized Exponential Families

第三章：小号：合作研究：Swaminathan Vishwanathan，Purdue University; Manfred Wardman，University of加州，Santa Cruz机器学习目前对于从海量数据集构建预测模型是不可或缺的。大多数广泛使用的机器学习算法都是基于最小化凸损失函数。所有这些模型的一个根本问题是，它们对离群值不鲁棒。为了解决这一限制，该项目开发了基于参数分布族（即t指数族）的概率模型，该模型导致准凸损失函数和对离群值具有鲁棒性的产量模型。在处理t-指数分布族时，与指数分布族的情况一样，关键的挑战是计算对数配分函数并有效地执行推理。该项目在两个具体案例中应对这一挑战。 对于少数类的问题，精确的迭代方案正在开发中。对于类的数量是指数大的问题，近似推理技术正在开发扩展变分方法。通过与Google的合作，该项目产生的一些数据挖掘算法正在应用于识别照片中的文本（PhotoOCR问题）的挑战性现实问题。该项目为研究生提供了以研究为基础的高级培训机会，也为本科生提供了机器学习和数据挖掘的研究机会。用于从在存在离群值的情况下稳健的数据构建预测模型的算法可能在广泛的应用中找到用途。该项目产生的算法、出版物和数据集的开源实现可通过项目网页获得：http://learning.stat.purdue.edu/wiki/tentropy/start

III: Small: Collaborative Research: Probabilistic Models using Generalized Exponential FamiliesSwaminathan Vishwanathan, Purdue University; Manfred Warmuth, University of California, Santa CruzMachine learning is currently indispensible for building predictive models from massive data sets. A large majority of widely used machine learning algorithms are based on minimizing a convex loss function. A fundamental problem with all such models is that they are not robust to outliers. To address this limitation, this project develops probabilistic models based on a parametric family of distributions, namely, the t-exponential family, that lead to quasi-convex loss functions and yield models that are robust to outliers. The key challenge when working with the t-exponential family of distributions, as in the case of the exponential family, is to compute the log-partition function and perform inference efficiently. The project addresses this challenge in two specific cases. For problems with small number of classes exact iterative schemes are being developed. For problems where the number of classes is exponentially large, approximate inference techniques are being developed by extending variational methods. In partnership with Google, some of the data mining algorithms resulting from this project are being applied to a challenging real-world problem of recognizing text in photos (the PhotoOCR problem). The project offers opportunities for research-based advanced training of graduate students as well as research opportuinities for undergraduates in machine learning and data mining. Algorithms for constructing predictive models from data that are robust in the presence of outliers are likely to find use in a broad range of applications. Open source implementions of algorithms, publications, and data sets resulting from the project are being made available through the project web page at: http://learning.stat.purdue.edu/wiki/tentropy/start