A New Generation of Trainable Machines for Multi-Task Learning

用于多任务学习的新一代可训练机器

• 批准号: EP/D071542/1
• 负责人: Massimiliano Pontil
• 金额: $ 97.68万
• 依托单位: University College London
• 依托单位国家: 英国
• 项目类别: Fellowship
• 财政年份: 2006
• 资助国家: 英国
• 起止时间: 2006 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FD071542%2F1
• 关键词: New; Generation; Trainable; Machines; Multi

The field of Machine Learning plays an increasingly important role in Computer Science and related disciplines. Over the past decade, the availability of powerful desktop computers has opened the door to synergistic interactions between empirical and theoretical studies of Machine Learning, showing thevalue of the ``learning from example'' paradigm in a wide variety of applications. Much effort has been devoted by Machine Learning researchers to the standard single task learning problem and exciting results have been derived. However, Machine Learning capabilities are still extremely limited when compared to those of humans. The human ability to generalise knowledge learned in one task in order to solve a new task is not available in current Machine Learning systems. Multi-task learning research has not yet received sufficient attention in the field. The standard single task learning approach builds on assumptions that are too restrictive to be easily extended to the novel learning scenarios which are envisaged in this proposal. Although interesting insights on multi-task learning have been provided, at present there is no comprehensive framework for multi-task learning and no cornerstone has yet been placed in the field. Thus, the main purpose of this proposal is to develop this area of Machine Learning research. The proposal focuses on Statistical Machine Learning methods for learning multiple related (classification or regression) tasks and integrating information across them. We shall design formal models of relationships between the tasks and develop (learning algorithms) for learning these relationships from data. We shall also develop the mathematical foundations (generalisation bounds, approximation results, convergence results) for multi-task learning, extending some key theoretical results for single tasklearning. Furthermore, the learning algorithms will be applied to two key applications, namely user preference modelling and multiple microarray gene expression data analysis. A central role in our approach is played by certain graph structures which allow us to model task relationships. This approach is very general and can be adapted to increasingly complex learning scenarios. The computational methods are based on the minimisation of certain penalty functionals via a large number of hyper-parameters associated with the tasks. The proposed research will lead to a new generation of trainable machines for multi-task learning, which will be more powerful and flexible models of learning, closer to human learning than previously developed Machine Learning frameworks.

机器学习在计算机科学及相关学科中发挥着越来越重要的作用。在过去的十年里，功能强大的台式计算机的出现为机器学习的经验研究和理论研究之间的协同互动打开了大门，显示了“从例子中学习”范例在各种应用中的价值。机器学习的研究者们对标准的单任务学习问题进行了大量的研究，并取得了令人振奋的结果。然而，与人类相比，机器学习的能力仍然非常有限。在当前的机器学习系统中，人类不具备概括在一个任务中学习的知识以解决新任务的能力。多任务学习的研究在该领域还没有得到足够的重视。标准的单一任务学习方法建立在过于严格的假设基础上，不容易扩展到本提案中设想的新的学习情景。虽然已经对多任务学习提出了有趣的见解，但目前还没有关于多任务学习的全面框架，也没有在这一领域放置基石。因此，这项建议的主要目的是发展这一领域的机器学习研究。该提案侧重于学习多个相关(分类或回归)任务并综合这些任务的信息的统计机器学习方法。我们将设计任务之间关系的正式模型，并开发(学习算法)从数据中学习这些关系。我们还将发展多任务学习的数学基础(泛化界、逼近结果、收敛结果)，推广单任务学习的一些关键理论结果。此外，学习算法将被应用于两个关键应用，即用户偏好建模和多微阵列基因表达数据分析。在我们的方法中，核心角色是某些图结构，这些图结构允许我们对任务关系进行建模。这种方法非常通用，可以适应日益复杂的学习场景。计算方法是基于通过与任务相关的大量超参数来最小化某些惩罚泛函。拟议的研究将导致用于多任务学习的新一代可训练机器，它将是更强大和灵活的学习模式，比以前开发的机器学习框架更接近人类学习。

项目成果

On Spectral Learning

• DOI: 10.5555/1756006.1756037
• 发表时间: 2010-03
• 期刊: J. Mach. Learn. Res.
• 作者: Andreas Argyriou;C. Micchelli;M. Pontil

When is there a representer theorem? Vector versus matrix regularizers

• DOI: 10.5555/1577069.1755870
• 发表时间: 2008-09
• 期刊: J. Mach. Learn. Res.
• 作者: Andreas Argyriou;C. Micchelli;M. Pontil

Taking Advantage of Sparsity in Multi-Task Learning

• 发表时间: 2009-03
• 期刊: arXiv: Machine Learning
• 作者: Karim Lounici;M. Pontil;A. Tsybakov;S. Geer

Representer Theorems for the matrix learning problem.

矩阵学习问题的表示定理。

• 发表时间: 2007
• 作者: C Micchelli