I-Corps: Using neural radiance fields (NeRF) and photogrammetry algorithms for creating 3D models

这个i-Corps项目的更广泛的影响/商业潜力是开发一种三维(3D)扫描技术，可以使用智能手机沉浸式地可视化空间。目前，全球有数百万人以内容创作者、观众、卖家和买家的身份运作。建议的技术提供了查看大型复杂对象和空间的能力。预计这一应用的第一个切入点是房地产电商。其目标是通过利用人工智能(AI)驱动的匹配、虚拟AR旅游和数字化租赁体验来简化房东和租户的租赁流程。这个i-Corps项目基于神经辐射场(NERF)和摄影测量算法的开发，以创建具有比仅有图像和视频可用的更多细节和信息的三维(3D)模型。目前，沉浸式技术非常繁琐，需要专业知识和特殊设备来构建远程体验。此外，基于纯摄影测量处理方法的3D重建算法需要大量数据(例如，一个房间需要1000张图像，而一座中等比例的历史建筑需要100,000张图像)。相比之下，由于NERF使用推理的能力(例如，一个房间需要100幅图像，进行3D扫描的时间大大减少)，所提出的方法通过NERF可以使用更小的数据集进行3D重建。从用户体验的角度来看，虽然图像和视频呈现了预定的内容，但3D模型可能允许用户简单地通过在复杂的空间中行走来进行探索。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of a three-dimensional (3D) scanning technology to visualize a space immersively using a smartphone. Currently, millions of people around the globe operate as content creators and viewers, sellers, and buyers. The proposed technology provides the ability to view large intricate objects and spaces. The expected first entry point for this application is e-commerce in real estate. The goal is to streamline the rental process for both landlords and tenants by leveraging artificial intelligence (AI)-driven matching, virtual AR tours, and digitizing the leasing experience. Additional applications are anticipated.This I-Corps project is based on the development of neural radiance fields (NeRF) and photogrammetry algorithms to create three-dimensional (3D) models with more detail and information than is available with images and videos alone. Currently, immersive technologies are cumbersome and require expertise and special equipment to build a remote experience. In addition, 3D reconstruction algorithms based on purely photogrammetry processing methods require large amounts of data (e.g., a room needs 1,000 images, while a medium scale historic building requires 100,000 images). In contrast, the proposed approach through NeRF enables 3D reconstruction with smaller data sets due to the ability of NeRF to use inference (e.g., a room needs 100 images and dramatically less time to 3D scan). From a user experience point of view, whereas images and videos present predetermined content, 3D models may allow users to simply explore by “walking through” complex spaces.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.