Collaborative Research: CIF: Small: Mathematical and Algorithmic Foundations of Multi-Task Learning

强化学习已经成为实时决策和控制的主要框架之一。它是最近几项引人注目的人工智能成功背后的推动力，在机器人控制、无线通信和蛋白质结构预测等各个领域都取得了成功。虽然强化学习为学习提供了一个强大而灵活的框架，但数据效率是一个根本性的挑战：众所周知，这个框架需要大量的计算资源和大量的数据。这一挑战限制了强化学习的适用性，并使其无法应用于训练数据和计算能力有限的问题，包括野火监测和使用无人驾驶飞行器搜索和救援失踪人员等重要应用。该项目通过开发多任务强化学习的新数学基础和新颖的学习算法来解决这一挑战，这些算法在联合学习多个任务时需要更少的汇总数据。该项目将研究成果与严格的教育和推广活动、课程开发和学生培训相结合。这个项目的重点是回答两个基本问题：(1)在什么条件下，联合学习多个任务比单独学习每个任务所需的数据和计算量更少？(2)强化学习算法能否在有限的数据和计算量下学习到有意义的东西？我们回答这些问题的方法借鉴了在线学习、压缩感知和随机建模等技术。特别是，该项目涵盖了离线设置和在线设置，其中任务之间的相似性结构是从给定的数据集中学习的，其中学习的结构用于有效地适应“动态”的新任务。该项目还解决了多任务学习中灾难性遗忘的基本问题，即学习策略在接受新任务训练后失去执行前一个任务的能力。在这个项目的过程中，提出的研究活动将通过一系列多机器人导航的模拟来系统地评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Reinforcement learning has emerged as one of the predominant frameworks for real-time decision making and control. It has been the driving force behind several recent high-profile successes of artificial intelligence, enjoying success in areas as diverse as robotic control, wireless communications, and protein structure prediction. While reinforcement learning provides a powerful and flexible framework for learning, data efficiency is a fundamental challenge: this framework is known to require significant computational resources and vast amount of data. This challenge limits the applicability of reinforcement learning and keeps it from being applied in problems where training data and computational power are limited, including important applications such as wildfire monitoring and the search-and-rescue of lost people using unmanned aerial vehicles. This project addresses this challenge by developing new mathematical foundations of multi-task reinforcement learning and novel learning algorithms that require less data in the aggregate when multiple tasks are jointly learned. The project integrates the research findings with rigorous educational and outreach activities, course development, and student training. This project focuses on answering two fundamental questions: (1) Under what conditions does it take less data and computation to learn multiple tasks jointly than it would to learn each task individually? and (2) Can reinforcement learning algorithms learn something meaningful with only a limited amount of data and computation? Our approach to answering these questions draws on techniques from online learning, compressed sensing, and stochastic modeling. In particular, this project covers both offline settings, where the similarity structure between tasks is learned from a given data set, and online settings, where this learned structure is used to efficiently adapt to a new task “on the fly”. The project also addresses the fundamental problem of catastrophic forgetting in multi-task learning, where the learned policy loses the ability to perform a previous task after training for a new task. Over the course of this project, the proposed research activities will be evaluated systematically through a series of simulations of multi-robot navigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.