Hybrid 4-Dimensional Augmented Reality Environments for Ubiquitous Markerless Context-Aware AEC/FM Applications

该项目的目标是测试PI提出的框架是否可以近实时地读取、写入和接收来自融合来自移动的设备和建筑信息模型的图片的模型的反馈，以便为建筑/工程/施工和设施管理（AEC/FM）应用提供无处不在和无标记的上下文感知。根据该框架，现场人员可以使用移动的设备来拍摄包括特定项目元素（例如，列），触摸或点击图像中的元素，并呈现（或能够添加）详细的信息列表，例如与物理元素相关的建筑/结构平面图。移动终端可以使用机载GPS和其他传感器来执行设备的视场和位置的粗略计算。初始图像处理在移动终端上完成，以提取特征点/描述符、视场和位置并将其发送到混合4维增强现实（HD 4AR）服务器。基于一种新的计算机视觉方法，服务器使用来自手机的这些信息来获得移动终端的位置，其分辨率比目前仅基于GPS的方法精确一个数量级。服务器使用导出的高精度相机位置来确定设备的相机所看到的网络信息。所提取的信息沿着每个网络信息项应该出现在照片中的像素坐标一起被返回到移动终端并以增强现实格式可视化。如果成功的话，这项研究的结果将为上下文感知应用提供第一个可行的平台，这些应用不需要可靠和高精度的基于GPS/传感器的位置和方向跟踪，并且基于现有的图像集合工作。它还通过增强现实形式的查询计划和实际现场信息的可视化来帮助现场人员，并支持项目人员和现场信息之间的互动。通过提供对信息的即时访问，拟议的框架自动提供来自实地活动的廉价、全球和频繁的报告，从而可以减少停机时间、返工、浪费和最终的成本超支。该项目还涉及教育和推广活动，以促进教学和学习，吸引本科生和研究生，并接触代表性不足的群体，K-12学生和行业专业人士。这些活动包括开发“民用基础设施工程和管理视觉传感”和“移动的网络物理系统”两个课程单元，以及为了解环境的AEC/FM应用程序创建新的软件工具和动手推广材料，这些材料将在研究和专业界广泛分发。

The objective of this project is to test whether a framework proposed by the PIs can in near real-time read, write, and receive feedback from a model which fuses pictures from mobile devices and Building Information Models for the purpose of providing ubiquitous and marker-less contextual awareness for Architecture/ Engineering/ Construction and Facility Management (AEC/FM) applications. According to the framework, field personnel can use mobile devices to take pictures that include specific project elements (e.g., column), touch or click on the elements in the image, and be presented with (or be able to add) a detailed list of information, such as architectural/structural plan related to the physical elements. The mobile device can use onboard GPS and other sensors to perform a rough calculation of the device's field-of-view and location. Initial image processing is done on the mobile device to extract and send feature points/descriptors, field-of-view, and location to the Hybrid 4-dimensional Augmented Reality (HD4AR) server. Based on a new computer vision method, the server uses this information from the phone to derive the mobile device's position at a resolution that is an order of magnitude more accurate than with current approaches based solely on GPS. The server uses the derived high-precision camera position to determine what cyber-information is in view of the device's camera. The extracted information, along with pixel coordinates of where each cyber-information item should appear in the photo, is returned to the mobile device and visualized in augmented reality format. If successful, the results of this research will provide the first feasible platform for context aware applications which does not require reliable and high accurate GPS/sensor-based location and orientation tracking and works based on existing image collections. It further assists field personnel through visualization of queried plan and actual site information in form of augmented reality, and supports interactions among project personnel and field information. By providing immediate access to information, the proposed framework automatically provides inexpensive, global and frequent reports from the field activities, and in turn can reduce downtime, rework, waste, and ultimately cost overrun. This project also involves educational and outreach activities to promote teaching and learning, engage undergraduate and graduate students, and reach out to underrepresented groups, K-12 students, and industry professionals. These activities include development of two course modules of "visual sensing for civil infrastructure engineering and management" and "mobile cyber-physical systems," as well as creating new software tools and hands-on outreach materials for context aware AEC/FM applications, which will be widely distributed among research and professional communities.