RI: Small: Object Detection, Pose Estimation, and Semantic Segmentation Using 3D Wireframe Models

人类可以识别图像中各种各样的对象类别，即使在遮挡和混乱中，并且尽管对象的形状，大小，外观和姿势存在显著的变化。虽然计算机视觉中的自动对象识别系统在过去十年中取得了很大进展，但最先进的方法主要依赖于对象类别的2D表示，这无法捕捉对象形状和姿态的大变化。该项目开发了对象的新3D表示，这些表示足够通用，可以描述各种对象类别的形状，但又足够简单，可以从2D图像集合中有效地学习，并用于检测和分割新图像中的对象。该项目可以影响许多研究领域，包括图像搜索，自主导航，医疗诊断工具，监控和机器人技术。该项目开发了一类新的3D线框模型，用于视图不变的目标检测，细粒度姿态估计和语义分割。线框模型是仅在3D对象边界上的几个点处定义的3D点、边和表面法线的稀疏集合。该项目开发了从多个对象类别的2D图像学习线框模型的方法，学习可变形线框模型的方法，这些模型捕获对象类别之间的类内形状变化，将外观信息集成到线框模型中的方法，以及基于线框模型的语义分割方法。

Humans can recognize a wide variety of object categories in an image, even in the midst of occlusion and clutter, and in spite of significant variability in the objects' shape, size, appearance, and pose. While automatic object recognition systems in computer vision have seen great progress over the past decade, state-of-the-art methods rely primarily on 2D representations of object categories, which cannot capture large variations in the objects' shape and pose. This project develops new 3D representations of objects that are general enough to describe the shape of a wide variety of object categories, yet simple enough so that they can be learned efficiently from a collection of 2D images, and used for detecting and segmenting objects in new images. This project can impact many research areas, including image search, autonomous navigation, medical diagnostic tools, surveillance and robotics.This project develops a new class of 3D wireframe models for view-invariant object detection, fine-grained pose estimation, and semantic segmentation. A wireframe model is a sparse collection of 3D points, edges and surface normals defined only at a few points on the boundaries of the 3D object. The project develops methods for learning wireframe models from 2D images of multiple object categories, methods for learning deformable wireframe models that capture intra-class shape variations across object categories, methods for integrating appearance information into wireframe models, and methods for semantic segmentation based on wireframe models.