PIPE: Probabilistic Models of Instructions, Perception andExperience - Representation, Learning and Reasoning

我们提出了联合贝叶斯模型的代表性，学习和推理的自主智能机器人复杂的推理任务的发展。这些任务包括：（1）自然语言中模糊表述的高级任务的指令解释和完成，（2）感知对象的检测、定位、分类和重构，（3）具有关于运行时任务参数化的动作特定执行推理的任务执行，可能的失败或不希望的影响，以改善自己的行为与每一个新的情况。概率模型目前使用的有希望的结果在上述三个子问题中。然而，这些通常是非常强烈的定制到各自的子问题，渲染知识的总体使用困难，如果不是不可能的，这个项目的目的是装备自主机器人系统的能力，建立概率知识库的经验对象的感知，自然语言的解释和物理执行的任务，并使用跨域获得的知识。为此，我们将开发统一的数据结构和算法，在一个全面的框架中利用表示，获取和推理。从而系统获得关于动作之间的关系、动作的效果、所涉及的对象和感知特性以及关于如何在不同的上下文中执行任务的信息的知识。我们在各个子领域的初步工作表明，以前独立的组件在机器人控制例程的协同作用，大大提高性能方面的自主性，通用性和多功能性的机器人。

We propose the development of joint Bayesian models for representation, learning and reasoning in sophisticated reasoning tasks in autonomous intelligent robotics. These tasks include: (1) the instruction interpretation and completion of vaguely formulated high-level tasks in natural language, (2) the detection, localization, categorization and reconstruction of perceived objects (3) the task execution with action-specific executive reasoning about task parameterization at runtime, possible failures or undesired effects to improve the own behavior with every new situation.Probabilistic models are currently used with promising results in all three of the above subproblems. However, these are usually very strongly tailored to the respective subproblem, rendering the overarching use of knowledge difficult, if not impossible.The aim of this project is to equip autonomous robotic systems with the ability to build up probabilistic knowledge bases from experiences of object perception, interpretation of natural language and the physical execution of tasks, and to use the gained knowledge across domains. To this end, we will develop unifying data structures and algorithms that leverage the representation, acquisition and reasoning in a comprehensive framework. Thereby the system gains knowledge about the relations between actions, their effects, involved objects and perceptual characteristics as well as information about how to perform a task in different contexts. Our preliminary work in the respective subareas demonstrates that the synergistic interaction of previously independent components in robotic control routines substantially boosts performance with respect to autonomy, generality and versatility of robots.