Semantic and Local Computer Vision based on Color/Depth Cameras in Robotics (SeLaVi)

对于现代机器人来说，识别环境中的物体是一项关键技能，可以实现有用和灵活的行动。为此，通常评估场景的一个或多个相机图像，并构建机器人环境的内部表示。然而，基于图像的目标识别面临着一些困难：一方面，几何简单和轻微纹理的对象，因为它们发生在许多应用中，几乎没有识别。另一方面，场景中（部分）隐藏的对象被感知得更差或根本不被感知。此外，感官信息通常仅描述各个对象的几何形状和位置，而不描述它们的语义功能或彼此之间的关系（例如，“A在B上”）。相比之下，感知作为一个长期目标，应该“理解”环境，以便通过机器人有意义地操纵复杂场景中的各种物体。此外，现场的几个局部视图应足以创建最完整的全球环境representations.The拟议的研究项目“SeLaVi”开发和研究新的概念，以图像为基础的理解场景。作为一种新的和独特的基本方法服务几何模型，它表示的对象由几个表面补丁（边界表示，BREPS），并从一个或多个深度图像生成。这确保了该方法的存储和计算效率显著高于使用公共点云或三角形网络的可能性。基于BRep和来自场景的附加颜色信息，识别对象数据库的对象。静态对象的识别应该在场景上具有很少的局部视图的情况下工作，并且对于其他移动对象（例如人类）尽可能鲁棒。以这种方式创建的世界模型，然后通过对象之间的语义关系进行扩展，以便能够通过机器人手臂进行操作。此外，考虑由用户半自动生成对象数据库。潜在的应用领域从自主服务机器人、编程到编程和人/机器人合作，到工业自动化（例如，盒子中的机器人）。

For modern robots, recognizing objects in their environment is a key skill that enables useful and flexible actions. For this purpose, usually one or more camera images of the scene are evaluated and an internal representation of the environment for the robot is built. However, image-based object recognition is confronted with some difficulties: on the one hand, geometrically simple and slightly textured objects, as they occur in many applications, are barely recognized. On the other hand, (partially) hidden objects in the scene are perceived worse or not at all. In addition, the sensory information usually describes only the geometry and location of the individual objects but not their semantic function or relationships with each other (e.g., "A is on B"). By contrast, perception as a long-term goal should "understand" the environment so that the various objects of a complex scene can be meaningfully manipulated by means of robots. In addition, few local views on the scene should be sufficient to create the most complete global environmental representation possible.The proposed research project "SeLaVi" develops and examines new concepts for image-based understanding of a scene. As a new and unique basic approach serve geometric models, which represent the objects by few surface patches (Boundary Representations, BReps) and which are generated from one or more depth images. This ensures a significantly higher storage and computational efficiency of the method than is possible with the common point clouds or triangular net-works. Based on the BRep and on additional color information from the scene, the objects of an object database are recognized. The recognition of the static objects should work with few local views on the scene and be as robust as possible against other moving objects (for example humans). The world model created in this way is then extended by semantic relations between the objects in order to enable manipulation by a robot arm. In addition, the semi-automatic generation of the object database by the user is considered. The potential fields of application range from autonomous ser-vice robots, programming-to-programming and human/robot cooperation, to industrial automation (e.g. handle-in-the-box).