Advancing the physical intelligence and performance of roBOTs towards human-like bi-manual objects MANipulation

点击翻译按钮获取中文摘要

The MANiBOT research aims at bi-manual mobile robots, able to perform a wide variety of manipulation tasks with highly diverse objects, possibly partly or fully unknown beforehand, in a human-like manner and performance. To achieve this, we advance and fuse all necessary technologies, from multimodal perception, cognition and control, to novel cognitive mechatronics. We develop new environment understanding and object/pose recognition methods, empowered through adaptive, context-aware fusion of vision, proximity and tactile sensing and emphasizing on adequate efficiency for fast and effective manipulation, even of objects without precise model, including deformable ones, and in diverse, challenging environments with human presence. We also develop a novel suite of manipulation primitives including non-prehensile manipulations, which along with bi-manual manipulation will allow the transfer of diverse objects with various sizes, weights, shapes, materials and rigidities from a mobile robot, with performance close to that of humans, even upon significant spatial constraints. The above are fused through a novel approach for robot cognitive functions based on multilevel robot cycles that allow learning, composing and swiftly adapting robot behaviors for complex manipulations, covering key topics of sequential manipulation of multiple objects to achieve complex goals. We push the limits of physical intelligence of bimanual mobile robots by coupling our methods with novel cognitive mechatronics, fusing advanced tactile and proximity sensors with a bi-manual mobile manipulator, optimized for energy efficiency and increased autonomy, including HRI capabilities for trustworthy and efficient operation. Our outcomes will be evaluated in four use cases, in three pilot sites (TRL5), in challenging environments where the handling of abundance of different objects is needed; in retail/supermarkets and transport/airports, for shelves restocking and baggage handling operations.