Semantic and Local Computer Vision based on Color/Depth Cameras in Robotics (SeLaVi)

对于现代机器人来说，识别环境中的物体是实现有用和灵活行动的关键技能。为此，通常会对场景的一个或多个相机图像进行评估，并为机器人构建环境的内部表示。然而，基于图像的物体识别面临着一些困难：一方面，在许多应用中，几何形状简单、纹理轻微的物体很难被识别。另一方面，场景中（部分）隐藏的物体会被认为更糟糕或根本不存在。此外，感官信息通常只描述单个物体的几何形状和位置，而不描述它们的语义功能或彼此之间的关系（例如，“A在B上”）。相比之下，感知作为一个长期目标应该“理解”环境，以便机器人能够有意义地操纵复杂场景中的各种物体。此外，很少有当地的观点，就足以创造出最完整的全球环境表现。拟议的研究项目“SeLaVi”开发并研究了基于图像的场景理解的新概念。几何模型是一种新的、独特的基本方法，它由一个或多个深度图像生成，用少量表面块表示物体（边界表示，BReps）。这确保了该方法比普通点云或三角形网络具有更高的存储和计算效率。基于BRep和来自场景的附加颜色信息，对对象数据库中的对象进行识别。静态对象的识别应该在场景中很少的局部视图下工作，并且对其他移动对象（例如人类）尽可能健壮。以这种方式创建的世界模型然后通过对象之间的语义关系进行扩展，以便通过机械臂进行操作。此外，还考虑了用户对对象数据库的半自动生成。潜在的应用领域包括自主服务机器人、编程对编程和人机合作，以及工业自动化（例如箱中手柄）。

For modern robots, recognizing objects in their environment is a key skill that enables useful and flexible actions. For this purpose, usually one or more camera images of the scene are evaluated and an internal representation of the environment for the robot is built. However, image-based object recognition is confronted with some difficulties: on the one hand, geometrically simple and slightly textured objects, as they occur in many applications, are barely recognized. On the other hand, (partially) hidden objects in the scene are perceived worse or not at all. In addition, the sensory information usually describes only the geometry and location of the individual objects but not their semantic function or relationships with each other (e.g., "A is on B"). By contrast, perception as a long-term goal should "understand" the environment so that the various objects of a complex scene can be meaningfully manipulated by means of robots. In addition, few local views on the scene should be sufficient to create the most complete global environmental representation possible.The proposed research project "SeLaVi" develops and examines new concepts for image-based understanding of a scene. As a new and unique basic approach serve geometric models, which represent the objects by few surface patches (Boundary Representations, BReps) and which are generated from one or more depth images. This ensures a significantly higher storage and computational efficiency of the method than is possible with the common point clouds or triangular net-works. Based on the BRep and on additional color information from the scene, the objects of an object database are recognized. The recognition of the static objects should work with few local views on the scene and be as robust as possible against other moving objects (for example humans). The world model created in this way is then extended by semantic relations between the objects in order to enable manipulation by a robot arm. In addition, the semi-automatic generation of the object database by the user is considered. The potential fields of application range from autonomous ser-vice robots, programming-to-programming and human/robot cooperation, to industrial automation (e.g. handle-in-the-box).