Linking metric and symbolic levels in autonomous reinforcement learning

强化学习（RL）已经成为自主智能体的理论基础，但在实际应用中很少使用。一个本质缺陷是维度的诅咒，我们在之前的SPP-1527资助的项目“大因子状态空间中的值表示”中解决了这个问题。然而，在我们的工作中，我们发现缺乏对新情况的适应能力同样是实际应用的障碍。我们的框架包含一个符号层，借用了关系强化学习，使度量转换模型适应新的情况。我们假设RL的度量层和符号层可以互补，以提供许多实际应用所需的精度和灵活性。为了研究我们开发的框架的可能性，我们正在追求以下科学目标：(1)开发一种贝叶斯方法来主动学习具有关系灵活性的度量转换模型；(2)研究度量和符号规划者层次中的协同作用，以改进规划并调整关系符号和行为。所设想的方法将把分解MDP和关系RL领域纳入一个公共框架，并允许自顶向下和自底向上的适应。我们期望在强烈依赖底层度量空间的关系任务（例如在机器人应用程序中）方面有重大改进。

Reinforcement learning (RL) has emerged as a well-founded theoretical basis for autonomous agents, but is rarely used in practical applications. One essential flaw is the curse of dimensionality, which we address in our previous SPP-1527 funded project "value representation in large factored state spaces". However, in our work we found the lack of adaptivity to new situations an equally obstructive obstacle for practical applications. Our framework contains a symbolic layer, borrowed from relational RL, to adapt the metric transitions model to new situations. We hypothesize that metric and symbolic layers of RL can work complementary to provide the precision and flexibility required in many practical applications. To investigate the possibilities of our developed framework we are pursuing the following scientific goals: (1) to develop a Bayesian method to actively learn a metric transition model with relational flexibility, and (2) to investigate synergies in the hierarchy of metric and symbolic planners to improve the planning and adjust the relational symbols and actions. The envisioned methods will cast the fields of factored MDP and relational RL into a common framework, and allow top-down and bottom-up adaptation. We expect significant improvement in relational tasks that depend strongly on an underlying metric spaces, for example in robotic applications.