Perception-guided robust and reproducible robotic grasping and manipulation

这项研究旨在开发和测试感知和操作策略，使机器人能够从复杂场景中抓取和操作对象，例如制造环境中的非结构化自遮挡反射金属部件堆，或核退役或混合废物回收中的未知/未建模和/或可变形的废物堆。该项目解决了电话会议中提到的关键挑战，即机器人使用由可模块化的子任务组成的新颖、独立于硬件的健壮技术来抓取和操纵对象。将制定可在不同硬件配置上重现的一般策略。事实上，从一开始，该项目就把重点放在稳健性和再现性上，这是联系所有项目目标的关键概念。该项目涉及的基本科学问题可概括如下：1)在复杂和困难的真实场景中进行稳健的可视数据收集、分割和产生一组可抓取的特征；2)基于抓取可见特征(而不是对象模型)和与硬件无关的抓取策略的抓取规划；3)抓取或重新抓取策略，以最好地实现所需的抓取后动作；以及也基于外部灵巧性，即利用环境或机器人的动态能力，以及4)将所有项目组件集成到将在所有参与者的实验室环境中实施的操作方案中。关于视觉数据收集和分析(项目1)，将开发能够在与不确定性有关的非结构化环境中工作的算法。该项目将解决各种工业应用所特有的困难环境。我们注意到，在视觉和机器人研究领域，工业基准数据集相对较少，尽管它们具有明显的经济重要性和显著的智力复杂性。在项目2中，将提出可抓取特征的概念，并将其用于设计新的抓取策略。还将制定评估操纵策略执行情况的手段，以评估所获得的结果的质量。通过使用可掌握特征的检测来管理感知-动作循环，该项目还将提供潜在地处理未知环境中的未知对象的工具。项目3遵循可抓取特征的概念，因为可抓取特征可能产生适当的临时抓取，但可能需要重新抓取，具体取决于要执行的任务。最后，项目组件的集成还会带来实施、实时约束和其他实际限制的问题。实验将在所有参与的研究小组中进行，最初使用相同或类似的设备，然后使用不同的设置和配置，以证明通用性、重复性和健壮性。这部作品的感知方面将专注于视觉上复杂、嘈杂和杂乱的场景。这项工作的操作方面将侧重于通用性和重复性，基于搜索可掌握的特征，而不是依赖对象模型。最后，该项目将产生大量数据，这些数据将被记录、共享并作为一套公共基准挑战提供给国际机器人研究界，包括培训和测试数据。

This research aims at developing and testing perception and manipulation strategies that will allow a robot to grasp and manipulate objects from a complex scene, e.g., an unstructured self-occluding heap of reflective metallic parts in manufacturing environments, or a heap of unknown/un-modeled and/or deformable waste materials in nuclear decommissioning or mixed waste recycling. The project addresses the key challenges mentioned in the call, namely, the grasping and manipulation of objects by robots using novel, hardware-independent, robust techniques, composed of modularisable subtasks. General strategies will be developed that will be reproducible on different hardware configurations. Indeed, from the outset, the project focuses on robustness and reproducibility, which are key concepts that connect all project objectives. The fundamental scientific questions addressed in the project can be summarized as follows: 1) Robust visual data collection, segmentation and production of sets of graspable features in complex and difficult real scenes, 2) Grasp planning based on grasping visible features (instead of object models) and hardware independent implementation of the grasping strategies, 3) Grasping or re-grasping strategies, to best enable desired post-grasp actions, and also based on extrinsic dexterity, namely the exploitation of the environment or the robot's dynamic capabilities, and 4) Integration of all project components into an operational scheme that will be implemented in the laboratory settings of all participants. Regarding visual data collection and analysis (item 1), algorithms capable of working in unstructured environments associated with uncertainty will be developed. The project will tackle difficult environments, which are characteristic of a variety of industrial applications. We note that industrial benchmark datasets are comparatively few in the vision and robotics research communities, despite their clear economic importance and also significant intellectual complexity. In item 2, the concept of graspable features will be developed and used to devise novel grasping strategies. Means of evaluating the performance of the manipulation strategies will also be developed in order to assess the quality of the results obtained. By managing the perception-action loop using the detection of graspable features, the project will also provide tools for potentially handling unknown objects in unknown environments. Item 3 follows the concept of graspable features since a graspable feature may yield a proper temporary grasp but may require re-grasping depending on the task to be performed. Finally, the integration of the project components will also raise issues of implementation, real-time constraints and other practical limitations. Experiments will be conducted in all participating research groups, initially using identical or similar equipment and then using different set ups and configurations in order to demonstrate generalisation, reproducibility and robustness. The perception aspect of the work will focus on visually complex, noisy and cluttered scenes. The manipulation aspect of the work will focus on generality and reproducibility, based on searching for graspable features rather than relying on object models. Finally, the project will generate a large amount of data, which will be logged, shared and made available to the international robotics research community as a set of public benchmark challenges, including training and testing data.