I-Corps: 3D Capturing Technology Based on Light Fields

I-Corps项目的广泛影响/商业潜力将有助于满足3D捕捉技术日益增长的需求，这主要是由于对3D应用的真实感内容的需求，以及对自动机械更好的计算机视觉解决方案的需求。此外，对于研究人员和保护主义者来说，3D捕捉是一种保护珍贵文物和研究它们的方法，而不会危及在运输过程中或由于自然退化而永久丢失它们。对于零售商和设计师来说，虚拟现实（VR）和增强现实（AR）应用程序需要低成本准确的产品3D表示，这些应用程序为与客户建立联系开辟了新的可能性。最后，随着显示器制造商不断改进光场（LF）显示器，LF图像和视频将有可能成为主流，并且需要存在一个管道来捕获、压缩和传输这种新形式的媒体。这个I-Corps项目利用专门的计算机数控相机和压缩算法来记录一个物体或一个场景的多张照片，并基于这些原始数据构建光场（LF）图像，这些图像保留了场景中灯光和表面之间复杂的相互作用。该项目开发了压缩和重建算法，利用LF图像中的冗余信息，同时允许在观看设备上快速解压（适用于web，虚拟现实（VR）和增强现实（AR））。3D内容（对象或场景）可以在六个自由度和立体视觉中观看。这提供了一个完整的解决方案，以捕获一个通用的对象，无论形状和材料在便携式3D格式，无需手动后期编辑。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project will help satisfy the growing demand in 3D capturing technology primarily caused by the need for photorealistic content for 3D applications, and better computer vision solutions for autonomous machinery. Additionally, for researchers and conservationists, 3D capturing is a way to preserve precious artifacts and study them without endangering of permanently losing them during transportation or due to natural deterioration. For retailers and designers, low cost accurate 3D representations of products are needed for virtual reality (VR) and augmented reality (AR) applications that open new possibilities to connect with their customers. Finally, as display makers continue to improve upon light field (LF) displays, LF image and video will potentially become mainstream and a pipeline will need to exist to capture, compress, and stream this new form of media.This I-Corps project utilizes a specialized computer numerical control camera and compression algorithms to record multiple photos of an object or a scene and based on this raw data to construct light field (LF) images which preserve the complex interactions between the lights and the surfaces in the scene. This project develops compression and reconstruction algorithms that exploit the redundant information found in LF images while also allowing quick decompression on viewing devices (applicable for web, virtual reality (VR), and augmented reality (AR)). The 3D content (object or scene) can then be viewed with six degrees of freedom and in stereovision. This provides a complete solution to capture a generic object regardless of shape and material in a portable 3D format without manual post-editing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.