Advancing the physical intelligence and performance of roBOTs towards human-like bi-manual objects MANipulation

MANiBOT研究的目标是双手移动机器人，能够以类似人类的方式和表现，对高度多样化的物体执行各种各样的操作任务，可能事先部分或完全未知。为了实现这一目标，我们推进并融合了所有必要的技术，从多模态感知、认知和控制，到新型认知机电一体化。我们开发了新的环境理解和对象/姿态识别方法，通过自适应，上下文感知的视觉，接近和触觉融合，并强调快速有效操作的足够效率，即使是没有精确模型的对象，包括可变形的对象，以及在人类存在的多样化，具有挑战性的环境中。我们还开发了一套新颖的操作原语，包括非握握性操作，它与双手动操作一起，将允许移动机器人转移具有各种尺寸，重量，形状，材料和刚度的各种物体，即使在显著的空间限制下，其性能也接近人类。以上内容通过基于多层机器人循环的机器人认知功能的新方法融合在一起，该方法允许学习，组合和快速适应复杂操作的机器人行为，涵盖了顺序操作多个对象以实现复杂目标的关键主题。我们通过将我们的方法与新颖的认知机电一体化相结合，将先进的触觉和接近传感器与双手移动机械手融合在一起，从而推动了双手移动机器人的物理智能极限，优化了能效和增强了自主性，包括可靠和高效操作的HRI功能。我们的结果将在四个用例中进行评估，在三个试验点（TRL5）中，在需要处理大量不同物体的具有挑战性的环境中；在零售/超市和运输/机场，用于货架补货和行李处理操作。

The MANiBOT research aims at bi-manual mobile robots, able to perform a wide variety of manipulation tasks with highly diverse objects, possibly partly or fully unknown beforehand, in a human-like manner and performance. To achieve this, we advance and fuse all necessary technologies, from multimodal perception, cognition and control, to novel cognitive mechatronics. We develop new environment understanding and object/pose recognition methods, empowered through adaptive, context-aware fusion of vision, proximity and tactile sensing and emphasizing on adequate efficiency for fast and effective manipulation, even of objects without precise model, including deformable ones, and in diverse, challenging environments with human presence. We also develop a novel suite of manipulation primitives including non-prehensile manipulations, which along with bi-manual manipulation will allow the transfer of diverse objects with various sizes, weights, shapes, materials and rigidities from a mobile robot, with performance close to that of humans, even upon significant spatial constraints. The above are fused through a novel approach for robot cognitive functions based on multilevel robot cycles that allow learning, composing and swiftly adapting robot behaviors for complex manipulations, covering key topics of sequential manipulation of multiple objects to achieve complex goals. We push the limits of physical intelligence of bimanual mobile robots by coupling our methods with novel cognitive mechatronics, fusing advanced tactile and proximity sensors with a bi-manual mobile manipulator, optimized for energy efficiency and increased autonomy, including HRI capabilities for trustworthy and efficient operation. Our outcomes will be evaluated in four use cases, in three pilot sites (TRL5), in challenging environments where the handling of abundance of different objects is needed; in retail/supermarkets and transport/airports, for shelves restocking and baggage handling operations.