Collaborative Research: CIF: Small: Mathematical and Algorithmic Foundations of Multi-Task Learning

强化学习已经成为实时决策和控制的主要框架之一。它一直是人工智能最近几次备受瞩目的成功背后的驱动力，在机器人控制，无线通信和蛋白质结构预测等领域取得了成功。虽然强化学习提供了一个强大而灵活的学习框架，但数据效率是一个根本性的挑战：众所周知，这个框架需要大量的计算资源和大量的数据。这一挑战限制了强化学习的适用性，并使其无法应用于训练数据和计算能力有限的问题，包括野火监测和使用无人机搜救失踪人员等重要应用。该项目通过开发多任务强化学习的新数学基础和新颖的学习算法来解决这一挑战，这些算法在联合学习多个任务时需要更少的数据。该项目将研究结果与严格的教育和推广活动，课程开发和学生培训相结合。 这个项目的重点是回答两个基本问题：（1）在什么条件下，联合学习多个任务比单独学习每个任务需要更少的数据和计算？（2）强化学习算法是否可以在有限的数据和计算量下学习到有意义的东西？我们回答这些问题的方法借鉴了在线学习，压缩传感和随机建模的技术。 特别是，该项目涵盖了离线设置，其中任务之间的相似性结构是从给定的数据集学习的，以及在线设置，其中这种学习的结构用于有效地适应新任务“在飞行中”。该项目还解决了多任务学习中灾难性遗忘的基本问题，即学习的策略在为新任务进行训练后失去了执行先前任务的能力。在该项目的过程中，将通过一系列多机器人导航的模拟系统地评估拟议的研究活动。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Reinforcement learning has emerged as one of the predominant frameworks for real-time decision making and control. It has been the driving force behind several recent high-profile successes of artificial intelligence, enjoying success in areas as diverse as robotic control, wireless communications, and protein structure prediction. While reinforcement learning provides a powerful and flexible framework for learning, data efficiency is a fundamental challenge: this framework is known to require significant computational resources and vast amount of data. This challenge limits the applicability of reinforcement learning and keeps it from being applied in problems where training data and computational power are limited, including important applications such as wildfire monitoring and the search-and-rescue of lost people using unmanned aerial vehicles. This project addresses this challenge by developing new mathematical foundations of multi-task reinforcement learning and novel learning algorithms that require less data in the aggregate when multiple tasks are jointly learned. The project integrates the research findings with rigorous educational and outreach activities, course development, and student training. This project focuses on answering two fundamental questions: (1) Under what conditions does it take less data and computation to learn multiple tasks jointly than it would to learn each task individually? and (2) Can reinforcement learning algorithms learn something meaningful with only a limited amount of data and computation? Our approach to answering these questions draws on techniques from online learning, compressed sensing, and stochastic modeling. In particular, this project covers both offline settings, where the similarity structure between tasks is learned from a given data set, and online settings, where this learned structure is used to efficiently adapt to a new task “on the fly”. The project also addresses the fundamental problem of catastrophic forgetting in multi-task learning, where the learned policy loses the ability to perform a previous task after training for a new task. Over the course of this project, the proposed research activities will be evaluated systematically through a series of simulations of multi-robot navigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.