Theoretical analysis of emerging machine learning paradigms

统计和计算机器学习领域取得了令人印象深刻的成功，为学习基于训练样本预测数据标签提供了严格的工具。支持向量机、决策树和Booking等算法范式从理论模型发展成为广受欢迎和广泛适用的软件包。因此，机器学习有力地证明了理论分析对实际应用开发的影响。然而，机器学习的许多常见应用解决了比当前可用的数学理论所建模的场景更复杂的场景。例如，考虑医学研究的目的是确定预测痴呆症未来发展的生理标记物。假设研究是在丹麦进行的。人们可以轻易地将其结论应用于加拿大患者吗？当训练和测试数据来自同一人群时，用于分类的常见学习方法执行得很好。然而，如上面示例所示，在许多应用程序中情况并非如此。在哪些条件下，以及如何使在某个源域上训练的分类器应用于不同的目标域？为了应对这种情况，学习从业者开发了启发式方法，尽管表面上工作得相当好，但现有的数学分析并不支持这种方法。机器学习理论分析的成功能否推广到这样的场景中？这项拟议的研究旨在为这些新兴的机器学习和数据挖掘启发式范例提供数学支持，这些范例虽然被广泛应用，但缺乏严格的理论基础。在最近的一系列论文中，我们和我的几个学生一起，在这个方向上迈出了开创性的一步。这类工作的一些例子包括：关于集群的流行模型选择启发式的论文(其中一篇获得COLT最佳学生论文)；NIPS‘06关于领域适应的论文；COLT08关于半监督学习的论文；以及NIPS’08关于集群的基础的论文。这些论文，以及我组织的相关研讨会，是我将用这笔赠款进行的一个雄心勃勃的项目的开始。

The field of statistical and computational machine learning has had impressive successes, offering rigorous tools for learning to predict labels of data, based on training samples. Algorithmic paradigms like Support-Vector-Machines, Decision-Trees and Boosting grew from theoretical models into popular and vastly applicable software packages. Machine learning thus provides a resounding demonstration of the impact of theoretical analysis on the development of practical applications. However, many common applications of machine learning address scenarios that are more complex than what is modeled by the currently available mathematical theory. Consider, for example, medical research aimed to identify physiological markers that predict future development of dementia. Say the research took place in Denmark. Can one readily apply its conclusions to Canadian patients? Common learning methods for classification perform well when training and test data are drawn from the same population. However, as the above example demonstrates, in many application this is not the case. Under which conditions, and how, can we adapt a classifier trained on some source domain to apply to a different target domain? To cope with such scenarios, learning practitioners have developed heuristics that, while apparently working reasonably well, are not supported by existing mathematical analysis. Can the success of theoretical analysis of machine learning be extended to such scenarios? The proposed research aims to provide mathematical support for such emerging machine learning and data mining heuristic paradigms that, while being widely applied, lack rigorous theoretical underpinnings. In a series of recent papers, with several students of mine, we have made pioneering steps in that direction. Some examples of such work include: papers on popular model-selection heuristics for clustering (one of which awarded COLT Best Student Paper); a NIPS'06 paper on domain adaptation; a COLT08 paper on semi-supervised learning, and a NIPS'08 paper on the foundations of clustering. These papers, as well as related workshops that I have organized, are the start of an ambitious project that I will pursue with this grant.