CHS: Medium: Nanophotonics Phased Array for Virtual and Augmented Reality Multifocal Displays

当前虚拟和增强现实(VR和AR)技术的爆炸式增长引起了学术界、工业界和大众的极大热情。不幸的是，这些技术也被证明会引发一系列严重的用户投诉，包括头痛、恶心和视力模糊。这对3D显示和内容来说尤其成问题，美国国家工程院已将“增强虚拟现实界面”确定为21世纪的一项重大挑战。研究表明，聚焦-适应冲突(或预期的“虚拟焦点”与显示器的实际焦面脱钩)是许多此类心理物理问题的原因。PI在这个项目中的目标是通过利用硅纳米光子学的进步来解决这种不匹配问题，开发一种自然感官的VR和AR多焦点显示的原型，以及多焦点和自动显示所需的视觉计算算法。PI还将设计和执行用户研究，以验证新技术(显示器和算法)。项目成果不仅将直接影响计算机图形学和可视化领域，而且还将影响一系列其他领域。大规模流体流动模拟的多焦点可调节-准确的立体可视化总体上将代表着科学和工程方面的突破，从人工动脉瓣膜的设计到飞行器，通过蛋白质对接的合理药物设计过程，以及可以利用实时MRI技术和其他高维医学成像数据的外科进步。同样重要的是新技术的潜在教育好处，它将释放VR和AR的力量，让年轻学生沉浸在他们通常无法进入的世界中，提供兴奋、参与以及在传统课堂环境中几乎不可能实现的背景的广度和深度。该项目的核心在于设计和实现一种新型的纳米光子相控阵芯片，它将能够通过大规模相控阵产生任意辐射图案，从而将相控阵的功能扩展到传统的光束聚焦和转向、通信和雷达之外，并将在图像处理、3D交互全息和虚拟现实方面开辟新的机会。该项目汇集了在3D计算机图形学和科学可视化、用于光的亚波长限制和转向的纳米光子学和等离子激元以及微电子、集成电路和微结构科学和技术方面具有丰富专业知识的合作者。这项研究将涉及三个方面：(A)慢光，以显著降低纳米光子学芯片的尺寸和功率要求；(B)分离芯片的电子和光学组件，以更好地优化每个组件；以及(C)开发高效算法，通过用户研究在傅立叶域中渲染和验证多焦点3D图形。多焦点显示器的设计是这项工作中风险最高、收益最高的部分；该小组制定了应急计划，以便在必要时继续开发多焦点和自动显示算法，并利用现有的多焦平面显示器进行用户研究。

The current explosion of virtual and augmented reality (VR and AR) technologies is the subject of much academic, industrial and popular enthusiasm. Unfortunately, these technologies have also been shown to induce a host of serious user complaints, including headaches, nausea and blurred vision. This is particularly problematic for 3D displays and content, and the National Academy of Engineering has identified "enhancing virtual reality interfaces" as a grand challenge for the 21st century. Studies have suggested that the vergence-accommodation conflict (or the decoupling of the intended "virtual focus" from the actual focal plane of a display) is the cause of many of these psychophysical problems. The PI's goal in this project is to address this mismatch by utilizing advances in silicon nanophotonics to develop a prototype natural-to-the-senses VR and AR multifocal display along with the requisite visual computing algorithms for multifocal and automultiscopic displays. The PI will also design and carry out user studies to validate the new technology (display and algorithms). Project outcomes will not only directly impact the fields of computer graphics and visualization, but an array of other fields as well. Multifocal accommodative-accurate stereo visualization of large-scale fluid flow simulations would represent a breakthrough in science and engineering generally, in areas ranging from the design of artificial arterial valves to aerial vehicles, the rational drug design process through protein docking, and in surgical advances that could take advantage of real-time MRI technologies and other high-dimensional medical imaging data. No less important are the potential educational benefits of the new technology, which would unleash the power of VR and AR to immerse young students in worlds to which they would normally not have access, providing excitement, engagement and a breadth and depth of context that is nearly impossible to achieve in a traditional classroom setting.The heart of this project lies in the design and implementation of a novel nanophotonic phased-array chip that will enable the generation of arbitrary radiation patterns with large-scale phased arrays, which would extend the functionality of phased arrays beyond conventional beam focusing and steering, communication and radar, and would open up new opportunities in image processing, 3D interactive holography, and virtual reality. This project brings together collaborators with significant expertise in 3D computer graphics and scientific visualization, in nanophotonics and plasmonics for the sub-wavelength confinement and steering of light, and in microelectronics, integrated circuits, and microstructure science and technology. The research will involve three thrusts: (a) slow light to significantly reduce the size and power requirements for the nanophotonics chip; (b) separate electronic and optical components of the chip to better optimize each; and (c) develop efficient algorithms to render and validate, through user studies, multifocal 3D graphics in the Fourier domain. The design of the multifocal display is the highest-risk and highest-gain component of the work; the team has contingency plans in place to proceed if necessary with developing multifocal and automultiscopic display algorithms, and for conducting user studies, employing existing multiple-focal-plane displays.