I-Corps: Using neural radiance fields (NeRF) and photogrammetry algorithms for creating 3D models

这个I-Corps项目更广泛的影响/商业潜力是开发三维（3D）扫描技术，使用智能手机身临其境地可视化空间。 当前，地球仪上有数百万人作为内容创建者和观看者、销售者和购买者进行操作。所提出的技术提供了查看大型复杂物体和空间的能力。 预计这一应用的第一个切入点是真实的房地产电子商务。 其目标是通过利用人工智能（AI）驱动的匹配、虚拟AR图尔斯之旅和数字化租赁体验，简化房东和租户的租赁流程。 这个I-Corps项目是基于神经辐射场（NeRF）和摄影测量算法的开发，以创建比单独使用图像和视频更详细和信息的三维（3D）模型。 目前，沉浸式技术很麻烦，需要专业知识和特殊设备来构建远程体验。 此外，基于纯粹摄影测量处理方法的3D重建算法需要大量数据（例如，一个房间需要1，000张图像，而一个中等规模的历史建筑需要100，000张图像）。相比之下，通过NeRF提出的方法由于NeRF使用推断（例如，一个房间需要100张图像，3D扫描的时间大大减少）。从用户体验的角度来看，图像和视频呈现的是预先确定的内容，而3D模型则可以让用户通过简单的“穿行”复杂空间来进行探索。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of a three-dimensional (3D) scanning technology to visualize a space immersively using a smartphone. Currently, millions of people around the globe operate as content creators and viewers, sellers, and buyers. The proposed technology provides the ability to view large intricate objects and spaces. The expected first entry point for this application is e-commerce in real estate. The goal is to streamline the rental process for both landlords and tenants by leveraging artificial intelligence (AI)-driven matching, virtual AR tours, and digitizing the leasing experience. Additional applications are anticipated.This I-Corps project is based on the development of neural radiance fields (NeRF) and photogrammetry algorithms to create three-dimensional (3D) models with more detail and information than is available with images and videos alone. Currently, immersive technologies are cumbersome and require expertise and special equipment to build a remote experience. In addition, 3D reconstruction algorithms based on purely photogrammetry processing methods require large amounts of data (e.g., a room needs 1,000 images, while a medium scale historic building requires 100,000 images). In contrast, the proposed approach through NeRF enables 3D reconstruction with smaller data sets due to the ability of NeRF to use inference (e.g., a room needs 100 images and dramatically less time to 3D scan). From a user experience point of view, whereas images and videos present predetermined content, 3D models may allow users to simply explore by “walking through” complex spaces.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.