Hybrid 4-Dimensional Augmented Reality Environments for Ubiquitous Markerless Context-Aware AEC/FM Applications

该项目的目的是测试pi提出的框架是否可以近乎实时地读取、写入和接收来自一个模型的反馈，该模型融合了来自移动设备和建筑信息模型的图片，目的是为建筑/工程/建筑和设施管理（AEC/FM）应用程序提供无处不在和无标记的上下文感知。根据该框架，现场人员可以使用移动设备拍摄包含特定项目元素（例如，列）的照片，触摸或点击图像中的元素，并呈现（或能够添加）详细的信息列表，例如与物理元素相关的建筑/结构平面图。移动设备可以使用车载GPS和其他传感器对设备的视野和位置进行粗略计算。初始图像处理在移动设备上完成，以提取并发送特征点/描述符、视野和位置到混合4维增强现实（HD4AR）服务器。基于一种新的计算机视觉方法，服务器使用来自手机的信息来获得移动设备的位置，其分辨率比目前仅基于GPS的方法精确一个数量级。服务器使用导出的高精度摄像机位置来确定设备摄像机所看到的网络信息。提取的信息，以及每个网络信息项目在照片中应该出现的像素坐标，被返回到移动设备，并以增强现实格式可视化。如果成功，这项研究的结果将为上下文感知应用提供第一个可行的平台，该应用不需要可靠和高精度的基于GPS/传感器的位置和方向跟踪，并基于现有的图像收集工作。通过增强现实的方式将查询到的计划和实际现场信息可视化，进一步协助现场人员，并支持项目人员与现场信息之间的交互。通过提供对信息的即时访问，所建议的框架自动提供来自现场活动的廉价的、全球性的和频繁的报告，并且反过来可以减少停机时间、返工、浪费和最终的成本超支。该项目还涉及教育和推广活动，以促进教学和学习，吸引本科生和研究生，并接触到代表性不足的群体，K-12学生和行业专业人士。这些活动包括开发“土木基础设施工程与管理的视觉传感”和“移动网络物理系统”两个课程模块，以及为上下文感知的AEC/FM应用创建新的软件工具和实践推广材料，这些材料将在研究和专业社区中广泛分发。

The objective of this project is to test whether a framework proposed by the PIs can in near real-time read, write, and receive feedback from a model which fuses pictures from mobile devices and Building Information Models for the purpose of providing ubiquitous and marker-less contextual awareness for Architecture/ Engineering/ Construction and Facility Management (AEC/FM) applications. According to the framework, field personnel can use mobile devices to take pictures that include specific project elements (e.g., column), touch or click on the elements in the image, and be presented with (or be able to add) a detailed list of information, such as architectural/structural plan related to the physical elements. The mobile device can use onboard GPS and other sensors to perform a rough calculation of the device's field-of-view and location. Initial image processing is done on the mobile device to extract and send feature points/descriptors, field-of-view, and location to the Hybrid 4-dimensional Augmented Reality (HD4AR) server. Based on a new computer vision method, the server uses this information from the phone to derive the mobile device's position at a resolution that is an order of magnitude more accurate than with current approaches based solely on GPS. The server uses the derived high-precision camera position to determine what cyber-information is in view of the device's camera. The extracted information, along with pixel coordinates of where each cyber-information item should appear in the photo, is returned to the mobile device and visualized in augmented reality format. If successful, the results of this research will provide the first feasible platform for context aware applications which does not require reliable and high accurate GPS/sensor-based location and orientation tracking and works based on existing image collections. It further assists field personnel through visualization of queried plan and actual site information in form of augmented reality, and supports interactions among project personnel and field information. By providing immediate access to information, the proposed framework automatically provides inexpensive, global and frequent reports from the field activities, and in turn can reduce downtime, rework, waste, and ultimately cost overrun. This project also involves educational and outreach activities to promote teaching and learning, engage undergraduate and graduate students, and reach out to underrepresented groups, K-12 students, and industry professionals. These activities include development of two course modules of "visual sensing for civil infrastructure engineering and management" and "mobile cyber-physical systems," as well as creating new software tools and hands-on outreach materials for context aware AEC/FM applications, which will be widely distributed among research and professional communities.