Automated As-Is Modeling with Visual and Spatial Data Fusion

本提案的目的是评估申请人提出的框架是否可以将建筑物内部重建为面向对象的模型。根据所提出的框架，建筑物内部首先调查与便携式视觉和空间传感系统，它提供了一个流的视频帧沿着与每像素的深度值。基于视频帧中的建筑物元素的视觉图案来识别它们。占用像素被标记有语义标签，诸如“列”或“墙”。同时，通过匹配它们的联合视觉特征的位置并评估特征的深度值来提取任意两个视频帧之间的逐点3D对应关系。基于这些3D对应关系，估计帧之间的成对相对变换。这样，帧中的像素可以对齐以形成3D点的“云”。根据先前分配给像素的标签，云中的点可以被划分为多个部分并进行参数化建模。这种面向对象的参数化模型代表了感知的建筑内部，并可用于建筑内部的施工和改造。

The objective of this proposal is to evaluate whether a framework proposed by the applicant can reconstruct building interiors into an object-oriented model. According to the proposed framework, building interiors are first surveyed with a portable visual and spatial sensing system, which provides a stream of video frames along with per-pixel depth values. Building elements in the video frames are recognized based on their visual patterns. The occupying pixels are tagged with semantic labels, such as "column" or "wall". Meanwhile, a set of point-wise 3D correspondences between any two video frames are extracted by matching the locations of their joint visual features and evaluating the features' depth values. Based on these 3D correspondences, the pair-wise relative transformation between the frames is estimated. This way, the pixels in the frames can be aligned to form a "cloud" of 3D points. According to the labels previously assigned to the pixels, the points in the cloud can be divided into multiple parts and modeled parametrically. This object-oriented parametric model represents the perceived building interiors, and can be used for building interior construction and retrofit.