RI: Medium: Learning MDP abstractions for Autonomous Systems using Variational Methods and Symmetry Groups

自动驾驶汽车、医院平台和家用机器人等自动驾驶系统具有巨大的潜在社会和经济效益，能够改变未来的工作、医疗保健和日常生活。然而，成功的自主性要求机器人能够做出自己的决定，并从自己的经验中学习。这可能是具有挑战性的，因为真实的世界是丰富和复杂的，自主机器人系统可能会被细节所迷惑。有时候，一个自治系统不会正确地概括：它会把两个非常相似的情况视为根本不同。该项目旨在开发新的方法来学习任务和领域适当的抽象，这将有助于自治系统更有效地推广到新的情况。更好的抽象将允许自治系统更有效地做出决策，从而改善学习和有效控制。本项目将研究马尔可夫决策过程和强化学习的决策理论框架内的抽象问题，这些框架已被广泛用作自动决策的框架。强化学习的最新进展使自主智能体和机器人能够完成具有挑战性的任务，有时甚至超过人类专家。然而，这在样本和计算复杂性方面都有极高的成本;即使在类似游戏的环境中，数百万个训练步骤和数天的训练时间也是典型的。这个项目将开发使这个过程更有效的方法，通过明确编码的目标，学习到代理的成本函数良好的抽象。具体而言，PI将研究和开发用于将大型连续决策问题压缩为小型离散决策问题的方法，以及将明确的对称约束编码到问题中的方法。这些方法将在不同复杂性的各种领域进行评估，包括涉及移动的导航和机器人操作的自主系统上的任务。总体目标是开发提高学习效率、抽象质量和对新任务和新情况的概括的方法。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Autonomous systems such as self-driving vehicles, hospital platforms, and household robots have great potential social and economic benefits, with the ability to transform the future of work, healthcare, and our daily routines. However, successful autonomy requires the robot to be able to make its own decisions and learn from its own experiences. This can be challenging because the real world is rich and complex and autonomous robotic systems can become confused by the details. It is sometimes the case that an autonomous system will not generalize properly: it will perceive two very similar situations to be fundamentally different. This project aims to develop new methods for learning task- and domain-appropriate abstractions that will help autonomous systems generalize to new situations more effectively. Better abstraction will allow autonomous systems to make decisions more efficiently leading to improved learning and effective control.This project will study the problem of abstraction within the decision-theoretic framework of Markov decision processes and reinforcement learning, which have been widely used as a framework for automated decision making. Recent advances in reinforcement learning have enabled autonomous agents and robots to accomplish challenging tasks, sometimes even surpassing human experts. However, this comes at an extremely high cost, both in sample and computational complexity; millions of training steps and days of training time are typical, even in game-like environments. This project will develop approaches for making this process much more efficient, by explicitly encoding objectives for learning good abstractions into the agent's cost function. Specifically, the PIs will study and develop approaches for compressing large continuous decision-making problems into small discrete ones, as well as approaches that incorporate explicit symmetry constraints that encode irrelevances in the problem. These methods will be evaluated on a variety of domains of varying complexity, including tasks on autonomous systems involving mobile navigation and robot manipulation. The overall objective is to develop approaches that improve learning efficiency, abstraction quality, and generalization to new tasks and situations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.