PIPE: Probabilistic Models of Instructions, Perception andExperience - Representation, Learning and Reasoning

提出了自主智能机器人复杂推理任务中表示、学习和推理的联合贝叶斯模型的发展。这些任务包括：(1)对自然语言中模糊表达的高级任务的指令解释和完成；(2)感知对象的检测、定位、分类和重构；(3)任务执行中使用特定于动作的执行推理，对运行时的任务参数、可能的失败或不期望的结果进行推理，以便在每一个新的情况下改善自己的行为。概率模型目前在上述三个子问题中都得到了良好的结果。然而，这些通常是非常强烈地为各自的子问题量身定做，使得知识的总体使用即使不是不可能的话也是困难的。本项目的目标是使自主机器人系统能够从对象感知、自然语言解释和任务的物理执行的经验中建立概率知识库，并使用所获得的跨域知识。为此，我们将开发统一的数据结构和算法，在一个全面的框架中利用表示、获取和推理。因此，系统获得关于动作之间的关系、它们的效果、涉及的对象和知觉特征的知识，以及关于如何在不同的上下文中执行任务的信息。我们在各自子领域的初步工作表明，机器人控制例程中先前独立的组件的协同交互大大提高了机器人在自主性、通用性和多功能性方面的性能。

We propose the development of joint Bayesian models for representation, learning and reasoning in sophisticated reasoning tasks in autonomous intelligent robotics. These tasks include: (1) the instruction interpretation and completion of vaguely formulated high-level tasks in natural language, (2) the detection, localization, categorization and reconstruction of perceived objects (3) the task execution with action-specific executive reasoning about task parameterization at runtime, possible failures or undesired effects to improve the own behavior with every new situation.Probabilistic models are currently used with promising results in all three of the above subproblems. However, these are usually very strongly tailored to the respective subproblem, rendering the overarching use of knowledge difficult, if not impossible.The aim of this project is to equip autonomous robotic systems with the ability to build up probabilistic knowledge bases from experiences of object perception, interpretation of natural language and the physical execution of tasks, and to use the gained knowledge across domains. To this end, we will develop unifying data structures and algorithms that leverage the representation, acquisition and reasoning in a comprehensive framework. Thereby the system gains knowledge about the relations between actions, their effects, involved objects and perceptual characteristics as well as information about how to perform a task in different contexts. Our preliminary work in the respective subareas demonstrates that the synergistic interaction of previously independent components in robotic control routines substantially boosts performance with respect to autonomy, generality and versatility of robots.