RoboSherlock - Scaling Robot Perception towards Everyday Manipulation through Unstructured Information Processing

我们提出了RoboSherlock的设计、实现和实证分析，RoboSherlock是一种新的机器人感知框架，用于促进人类生活环境中日常使用对象的日常操作活动。RoboSherlock项目的目标是为机器人配备灵活、准确和强大的感知系统，而不必中断正在进行的活动。这是通过为感知系统配备以下能力来实现的：-终身学习和适应日常活动，-强大的一阶概率推理，-基于知识和模型的感知方法，以及-主动和预期感知。为了实现RoboSherlock，我们将感知视为一个非结构化的信息处理和管理问题。非结构化信息管理（UIM）已被证明是一种强大的范例，可以将智能信息和问答系统扩展到现实世界的复杂性。UIM中的复杂性是通过在非结构化文档中识别（或假设）结构化信息片段、应用专家组合对信息片段进行注释、测试并将这些孤立的注释集成到文档的综合解释中来处理的。在RoboSherlock项目中，我们将把这些方法转移到机器人感知中，以便将其扩展到现实环境中的现实感知任务。在RoboSherlock中实现的感知能力将在一个机器人代理的背景下进行评估和经验分析，该机器人代理每周执行5天的日常活动取取和放置任务，持续数周，例如设置桌子，清理，装卸洗碗机等。

We propose the design, implementation, and empirical analysis of RoboSherlock, a novel robot perception framework for facilitating everyday manipulation activities with objects of daily use in human living environments. The aim of the RoboSherlock project is to equip robots with perception systems that are flexible, accurate, and robust without having to interrupt ongoing activity. This is achieved by equipping the perception system with the following capabilities:- lifelong learning and adaptation for everyday activity,- powerful first-order probabilistic reasoning,- knowledge- and model-based perception methods, and- proactive and anticipatory perception.In order to implement RoboSherlock we consider perception as an unstructured information processing and management problem. Unstructured information management (UIM) has proven itself to be a powerful paradigm for scaling intelligent information and question answering systems towards real-world complexity. Complexity in UIM is handled by identifying (or hypothesizing) pieces of structured information in unstructured documents, by applying ensembles of experts for annotating information pieces, and by testing and integrating these isolated annotations into a comprehensive interpretation of the document. In the RoboSherlock project we will transfer these methods to robot perception in order to scale it to realistic perception tasks in realistic environments.The perception capabilities implemented in RoboSherlock will be evaluated and empirically analyzed in the context of a robotic agent that performs fetch and place tasks for everyday activities, 5 days a week for several weeks, such as setting the table, cleaning up, loading and unloading the dishwasher, etc.