RI: Small: Object Detection, Pose Estimation, and Semantic Segmentation Using 3D Wireframe Models

人类可以识别图像中各种各样的对象类别，即使在遮挡和杂乱中，尽管对象的形状、大小、外观和姿势存在显著变化。虽然计算机视觉中的自动目标识别系统在过去的十年中取得了很大的进步，但最先进的方法主要依赖于目标类别的2D表示，这无法捕捉到目标形状和姿势的巨大变化。这个项目开发了新的对象的3D表示，这些表示足够通用，可以描述各种对象类别的形状，但又足够简单，以至于可以从2D图像集合中有效地学习它们，并用于检测和分割新图像中的对象。该项目将影响图像搜索、自主导航、医疗诊断工具、监控和机器人等多个研究领域。该项目开发了一类新的三维线框模型，用于视觉不变目标检测、细粒度姿态估计和语义分割。线框模型是仅在3D对象边界上的几个点上定义的3D点、边和曲面法线的稀疏集合。该项目开发了用于从多个对象类别的2D图像中学习线框模型的方法、用于学习捕捉跨对象类别的类内形状变化的可变形线框模型的方法、用于将外观信息集成到线框模型中的方法以及用于基于线框模型的语义分割的方法。

Humans can recognize a wide variety of object categories in an image, even in the midst of occlusion and clutter, and in spite of significant variability in the objects' shape, size, appearance, and pose. While automatic object recognition systems in computer vision have seen great progress over the past decade, state-of-the-art methods rely primarily on 2D representations of object categories, which cannot capture large variations in the objects' shape and pose. This project develops new 3D representations of objects that are general enough to describe the shape of a wide variety of object categories, yet simple enough so that they can be learned efficiently from a collection of 2D images, and used for detecting and segmenting objects in new images. This project can impact many research areas, including image search, autonomous navigation, medical diagnostic tools, surveillance and robotics.This project develops a new class of 3D wireframe models for view-invariant object detection, fine-grained pose estimation, and semantic segmentation. A wireframe model is a sparse collection of 3D points, edges and surface normals defined only at a few points on the boundaries of the 3D object. The project develops methods for learning wireframe models from 2D images of multiple object categories, methods for learning deformable wireframe models that capture intra-class shape variations across object categories, methods for integrating appearance information into wireframe models, and methods for semantic segmentation based on wireframe models.