Automated As-Is Modeling with Visual and Spatial Data Fusion

本提案的目的是评估申请人提出的框架是否可以将建筑内部重建为面向对象的模型。根据提出的框架，首先使用便携式视觉和空间传感系统对建筑内部进行调查，该系统提供视频帧流以及每像素深度值。视频帧中的建筑元素是根据它们的视觉模式来识别的。占用像素用语义标签标记，如“列”或“墙”。同时，通过匹配任意两个视频帧的联合视觉特征的位置并评估特征的深度值，提取任意两个视频帧之间的一组逐点的3D对应关系。基于这些三维对应，估计了帧之间的成对相对变换。这样，帧中的像素就可以对齐形成3D点的“云”。根据先前分配给像素的标签，可以将云中的点划分为多个部分并进行参数化建模。这种面向对象的参数化模型表示感知到的建筑内部，可用于建筑内部的构造和改造。

The objective of this proposal is to evaluate whether a framework proposed by the applicant can reconstruct building interiors into an object-oriented model. According to the proposed framework, building interiors are first surveyed with a portable visual and spatial sensing system, which provides a stream of video frames along with per-pixel depth values. Building elements in the video frames are recognized based on their visual patterns. The occupying pixels are tagged with semantic labels, such as "column" or "wall". Meanwhile, a set of point-wise 3D correspondences between any two video frames are extracted by matching the locations of their joint visual features and evaluating the features' depth values. Based on these 3D correspondences, the pair-wise relative transformation between the frames is estimated. This way, the pixels in the frames can be aligned to form a "cloud" of 3D points. According to the labels previously assigned to the pixels, the points in the cloud can be divided into multiple parts and modeled parametrically. This object-oriented parametric model represents the perceived building interiors, and can be used for building interior construction and retrofit.