CHS: Small: Towards Photorealistic Augmented and Virtual Reality Displays

3D显示的圣杯是产生一种在我们看来与现实难以区分的场景。尽管显示技术取得了重大进步，但对所有感知深度线索的同时欺骗仍然超出了大多数显示器的能力范围。因此，现有的3D显示器，特别是那些用于增强/虚拟现实(AR/VR)应用程序的显示器，往往会由于现实世界和虚拟世界之间的细微差异而导致不适。该项目旨在开发新颖的3D照片真实感显示设计，确保入射到眼睛上的光场以比现有设计高得多的保真度模拟现实。这种技术的发展将对AR/VR系统和应用以外的广泛学科和应用产生影响，如眼科和光学系统测试。此外，随着显示内容的视觉真实感变得更好，它们将取代如今在工作和家庭环境中无处不在的标准2D显示器。这些逼真的显示器将有可能使AR/VR系统更容易为视障人士所用。该项目的教育和推广部分通过实验室访问和动手工作室在大匹兹堡地区的初中和高中传播显示研究和演示。为了实现照片逼真的AR/VR显示的目标，该项目探索了不同的虚拟世界建模方法，并重点开发了三种不同的方法，以牺牲复杂性为代价逐步增加真实感：(I)以深度和纹理分解的形式的简单模型；(Ii)提供密集光线空间采样的光场表示；(Iii)使用傅里叶光学的相干波前。这三种方法都旨在产生空间和角度分辨率高的光场，尽管基本的物理机制依赖于不同的光传输模型。该项目将开发生成易于适应显示要求的内容的技术。该项目还探索和表征了可实现的空间和角度分辨率、遮挡描述的精确度、眼箱的大小以及它们的限制方面的基本属性和特征。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The holy grail for a 3D display is to produce a scene that, to our eyes, is indistinguishable from reality. Despite significant advances in display technology, simultaneous deception of all perceptual depth cues is still beyond the reach of most displays. As a consequence, existing 3D displays, especially those used in Augmented / Virtual Reality (AR/VR) applications, tend to cause discomfort due to subtle differences between the real and the virtual world. This project aims to develop novel 3D photorealistic display designs that ensure that the light field incident on the eye mimics reality at a significantly higher level of fidelity as compared to existing designs. The development of such a technology stands to impact a broad range of disciplines and applications that go beyond AR/VR systems and applications, such as ophthalmology, and testing of optical systems. Further, as the visual realism of displayed content gets better, they stand to replace standard 2D monitors that are ubiquitous in work and home environments today. These photorealistic displays will have the potential to make AR/VR systems more accessible for visually impaired people. The education and outreach components of this project disseminates display research and demos in middle and high schools in the greater Pittsburgh region via lab visits, and hands-on workshops.To achieve the goal of photorealistic AR/VR displays, this project explores differ approaches for modeling of the virtual world and focuses on the development of three distinct approaches that progressively increase realism at the cost of complexity: (i) simple models in form of depth and texture decomposition; (ii) light field representations that provide a dense sampling of the ray space; (iii) coherent wavefronts using Fourier optics. All the three approaches aim to produce light fields that are dense in spatial and angular resolution even though the underlying physical mechanisms rely on different models for light transport. The project will develop techniques for generating content that is easily adapted to the requirements of the display. The project also explores and characterizes the fundamental properties and features in terms of achievable spatial and angular resolutions, the precision in depiction of occlusion, the size of the eye-box, and finally their limitations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.