NRI: Collaborative Research: A Framework for Hierarchical, Probabilistic Planning and Learning

这个项目致力于创建一个统一的框架来解决具有不确定状态和动作的非常大的问题，例如在真实世界环境中操作的机械手机器人。这一结果可能会对辅助技术带来特别大的希望，包括可以被老年人和残疾人使用的自主机器人，以帮助他们进行日常活动。拟议的集成框架将表示、应用和学习分层的领域知识，并将包括将知识从较简单的问题转移到较复杂的问题的能力。这项研究将使自主代理能够根据经验开发复杂领域的结构化表示。代理将使用学习的表示法来解释低级和高级请求的自然语言命令。技术重点是通过在多个抽象级别生成和利用概率领域知识，在大型、不确定的领域实现易于处理的规划。代理将自主地创建分层表示，其中的层相互构建以产生复杂的行为。代理将学习执行有用的行为，例如使用低级别传感器反馈导航或组装复杂对象，如桥梁或桌子。关键的技术贡献将是用于(1)使用抽象的面向对象的马尔可夫决策过程(AMDP)模型在大的状态/动作空间中进行规划的方法，该模型是在多个抽象层次上表示概率领域知识的新形式；(2)以AMDP的形式学习分层任务知识；以及(3)通过映射到所学习的分层结构来解释多个抽象层次上的自然语言命令。这种形式主义将在几个领域得到演示和验证，包括模拟的“清理”玩具领域，具有挑战性的复杂视频游戏，以及机器人操作任务。

This project is an effort to create a unified framework for solving very large problems with uncertain states and actions, such as manipulator robots acting in real-world environments. The results may have especially great promise for assistive technologies, including autonomous robots that can be used by elderly and disabled populations to aid them in their daily activities. The proposed integrated framework will represent, apply, and learn hierarchical domain knowledge, and will include the ability to transfer knowledge from simpler problems to more complex ones. The research will enable autonomous agents to develop a structured representation of complex domains based on experience. The agents will use learned representations to interpret natural language commands for both low-level and high-level requests. The technical focus is enabling tractable planning in large, uncertain domains by generating and leveraging probabilistic domain knowledge at multiple levels of abstraction. Agents will autonomously create layered representations in which the layers build on one another to produce complex behaviors. Agents will learn to perform useful behaviors, such as navigating using low-level sensor feedback or assembling complex objects such as a bridge or a table. The key technical contributions will be methods for (1) planning in large state/action spaces using the abstract object-oriented Markov decision process (AMDP) model, a new formalism for representing probabilistic domain knowledge at multiple levels of abstraction; (2) learning hierarchical task knowledge in the form of AMDPs; and (3) interpreting natural language commands at multiple levels of abstraction by mapping to the learned hierarchical structure. The formalism will be demonstrated and validated in several domains, including a simulated "cleanup" toy domain, challenging and complex video games, and a robot manipulation task.