RI: Small: 3D Nonrigid Object Reconstruction from Large-Scale Unorganized 2D Images

从多幅二维图像重建物体的三维形状是计算机视觉中的一个基本问题。关于这个问题的先前工作通常要求感兴趣的对象是刚性的，或者可用的2D图像被很好地组织，例如视频中的连续帧。该项目研究了从大量无组织的2D图像中重建非刚性3D对象的挑战性问题，这些图像可能是在不同时间，不同背景，不同视角，不同光照条件下和/或使用不同相机拍摄的。研究团队开发了结合对象定位，特征匹配，以及跨所述图像的部分形状匹配，以从所述输入图像分割所述2D感兴趣对象。分割的2D对象被组织成簇，以恢复底层的3D非刚性变形。从这些聚类中重建3D对象的片段，并最终通过去除中间的非刚性变形来组装以获得完整的3D对象。该研究成果在计算机视觉、计算机图形学、计算机游戏、动物学、微生物学、海洋科学和医学研究等领域都有广泛的应用价值，这些领域都涉及到三维物体的建模。在目标定位、特征匹配和部分形状匹配方面取得的进展在目标检测、目标识别、图像搜索、监视、跟踪和分割等方面有着直接的应用。这项研究也为本科生和研究生的培养提供了一个很好的环境。

Reconstructing the 3D shape of an object from multiple 2D images is a fundamental problem in computer vision. Prior work on this problem usually requires the object of interest to be rigid or the available 2D images to be well organized, such as consecutive frames in a video. This project investigates the challenging problem of reconstructing a nonrigid 3D object from a large number of unorganized 2D images, which may be taken at different times, with different backgrounds, from different perspectives, under different lighting conditions, and/or using different cameras.The research team develops new algorithms of combining object localization, feature matching, and partial shape matching across the images to segment the 2D object of interest from the input images. The segmented 2D objects are organized into clusters to recover the underlying 3D nonrigid deformation. Pieces of the 3D object are reconstructed from these clusters and finally assembled to obtain the complete 3D object by removing the in-between nonrigid deformations. An image database with 2D images of selected nonrigid objects is constructed for performance evaluation.This research benefits many applications in computer vision, computer graphics, computer gaming, zoology, microbiology, marine science, and medical research, which all involve the modeling of 3D norigid objects. Progress made on object localization, feature matching and partial shape matching has immediate applications in object detection, object recognition, image search, surveillance, tracking, and segmentation. This research also provides an excellent setting for the training of both undergraduate and graduate students.