Tighter error bounds for representation learning and lifelong learning

表征学习和终身学习的更严格的误差范围

• 批准号: RGPIN-2018-03942
• 负责人: Mehta, Nishant
• 金额: $ 2.84万
• 依托单位: University of Victoria
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=687450
• 关键词: Tighter; error; bounds; representation; learning

The success of machine learning algorithms crucially hinges on which numerical features are used to represent data. An arguably large part of the success of human learning is that we not only learn representations to perform well on prediction tasks, but we also reuse these representations to more efficiently learn newly encountered but similar tasks. We are now in an era where "deep" machine learning methods actually learn useful representations of data automatically. These deep learning methods are becoming vital to many other fields: just a few successful applications include drug design, 3D rendering for computer graphics, and beating top-level human players in games such as Go. Despite these great empirical successes, our mathematical understanding for why these methods work well and how to best train them is lacking. ******Part I of my research is to develop new theory to understand how well a deep model will perform when it makes predictions about new data. My strategy to obtain better performance guarantees is to use theory that leverages specific properties of the actual data an algorithm sees and specific properties of the algorithm itself. By considering both of these important aspects of training, I expect to achieve success for deep learning models as well. Another important part of my analysis is to answer the following question: If a deep learning model is insensitive to certain types of transformations of its input, can it successfully be trained using less data as a result?******One of the greatest advancements the field of machine learning can make is to shift from learning each new thing in isolation to reusing what has been learned in the past when learning new tasks. This continual transfer when learning an endless sequence of tasks is known as lifelong learning. While some research has begun in this important area, the mathematical theory for how well algorithms can perform lifelong learning is lacking, especially with regards to adaptive algorithms that transfer much more from their past experiences when they encounter tasks that are highly similar to past tasks. ******Part II of my research is to design algorithms for lifelong learning, including developing mathematical guarantees for these algorithms. I aim to answer questions like the following: If a learning agent encounters a series of slowly changing learning tasks, can we pool the data from previous tasks in order to learn new tasks using much less data? My strategy will be to adapt powerful ideas from sequential prediction, a vital yet previously untapped resource for work in lifelong learning.******Both parts of this work can fundamentally advance the field of machine learning, which already is revolutionizing a number of sciences and industries. The students participating in this research will be excellently-equipped to fuel Canada's technological contributions at the global stage.

机器学习算法的成功关键取决于使用哪些数字特征来表示数据。可以说，人类学习成功的很大一部分是，我们不仅学习表征来很好地执行预测任务，而且我们还重复使用这些表征来更有效地学习新遇到的类似任务。我们现在所处的时代，“深度”机器学习方法实际上会自动学习有用的数据表示。这些深度学习方法正变得对许多其他领域至关重要：仅有几个成功的应用程序，包括药物设计、计算机图形的3D渲染，以及在围棋等游戏中击败顶级人类棋手。尽管取得了这些巨大的经验上的成功，但我们对这些方法为什么有效以及如何最好地训练它们缺乏数学上的理解。*我研究的第一部分是开发新的理论，以了解深度模型在对新数据进行预测时的表现如何。为了获得更好的性能保证，我的策略是使用利用算法看到的实际数据的特定属性和算法本身的特定属性的理论。通过考虑培训的这两个重要方面，我希望深度学习模型也能取得成功。我分析的另一个重要部分是回答以下问题：如果深度学习模型对其输入的某些类型的转换不敏感，是否可以使用更少的数据来成功地训练它？*机器学习领域可以取得的最大进步之一是，在学习新任务时，从孤立地学习每一件新事物转向重复使用过去学到的东西。当学习一系列无穷无尽的任务时，这种持续的转移被称为终身学习。虽然在这一重要领域已经开始了一些研究，但缺乏关于算法能够多好地执行终身学习的数学理论，特别是关于自适应算法，当它们遇到与过去任务高度相似的任务时，这些算法更多地从过去的经验中转移出来。*我研究的第二部分是为终身学习设计算法，包括为这些算法开发数学保证。我的目标是回答如下问题：如果学习代理遇到一系列变化缓慢的学习任务，我们是否可以将以前任务的数据汇集在一起，以便使用更少的数据学习新任务？我的战略将是采用来自序列预测的强大想法，这是一种重要的但以前尚未开发的资源，用于终身学习。这项工作的两个部分都可以从根本上推动机器学习领域的发展，该领域已经在给许多科学和行业带来革命性的变化。参与这项研究的学生将具备出色的装备，以推动加拿大在全球舞台上的技术贡献。