CHS: Small: Towards Photorealistic Augmented and Virtual Reality Displays

3D显示器的终极目标是创造一个在我们看来与现实无法区分的场景。尽管显示技术取得了重大进展，但大多数显示器仍然无法同时欺骗所有感知深度线索。因此，现有的3D显示器，特别是用于增强/虚拟现实（AR/VR）应用的3D显示器，由于真实世界和虚拟世界之间的细微差异，往往会引起不适。该项目旨在开发新颖的3D逼真显示设计，确保入射到眼睛上的光场以比现有设计更高的保真度模拟现实。这种技术的发展将影响广泛的学科和应用，超越AR/VR系统和应用，如眼科和光学系统测试。此外，随着显示内容的视觉真实感越来越好，它们将取代目前在工作和家庭环境中无处不在的标准2D显示器。这些逼真的显示器将有可能使视障人士更容易使用AR/VR系统。该项目的教育和推广部分通过实验室参观和实践研讨会在大匹兹堡地区的初中和高中传播展示研究和演示。为了实现逼真的AR/VR显示的目标，本项目探索了虚拟世界建模的不同方法，并着重于三种不同方法的发展，以复杂性为代价逐步增加真实感：(i)以深度和纹理分解形式的简单模型；（ii）提供光线空间密集采样的光场表示；（iii）使用傅立叶光学的相干波前。所有这三种方法的目标都是产生空间和角度分辨率密集的光场，尽管潜在的物理机制依赖于光传输的不同模型。该项目将开发生成内容的技术，这些内容很容易适应显示的要求。该项目还探索和描述了可实现的空间和角度分辨率、遮挡描述的精度、眼框的大小以及它们的局限性等方面的基本属性和特征。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The holy grail for a 3D display is to produce a scene that, to our eyes, is indistinguishable from reality. Despite significant advances in display technology, simultaneous deception of all perceptual depth cues is still beyond the reach of most displays. As a consequence, existing 3D displays, especially those used in Augmented / Virtual Reality (AR/VR) applications, tend to cause discomfort due to subtle differences between the real and the virtual world. This project aims to develop novel 3D photorealistic display designs that ensure that the light field incident on the eye mimics reality at a significantly higher level of fidelity as compared to existing designs. The development of such a technology stands to impact a broad range of disciplines and applications that go beyond AR/VR systems and applications, such as ophthalmology, and testing of optical systems. Further, as the visual realism of displayed content gets better, they stand to replace standard 2D monitors that are ubiquitous in work and home environments today. These photorealistic displays will have the potential to make AR/VR systems more accessible for visually impaired people. The education and outreach components of this project disseminates display research and demos in middle and high schools in the greater Pittsburgh region via lab visits, and hands-on workshops.To achieve the goal of photorealistic AR/VR displays, this project explores differ approaches for modeling of the virtual world and focuses on the development of three distinct approaches that progressively increase realism at the cost of complexity: (i) simple models in form of depth and texture decomposition; (ii) light field representations that provide a dense sampling of the ray space; (iii) coherent wavefronts using Fourier optics. All the three approaches aim to produce light fields that are dense in spatial and angular resolution even though the underlying physical mechanisms rely on different models for light transport. The project will develop techniques for generating content that is easily adapted to the requirements of the display. The project also explores and characterizes the fundamental properties and features in terms of achievable spatial and angular resolutions, the precision in depiction of occlusion, the size of the eye-box, and finally their limitations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.