CHS: Medium: Nanophotonics Phased Array for Virtual and Augmented Reality Multifocal Displays

当前虚拟现实和增强现实（VR和AR）技术的爆炸式发展引起了学术界、工业界和大众的极大热情。不幸的是，这些技术也被证明会引发一系列严重的用户投诉，包括头痛、恶心和视力模糊。对于3D显示和内容来说，这尤其是个问题，美国国家工程院（National Academy of Engineering）已经将“增强虚拟现实界面”确定为21世纪的重大挑战。研究表明，趋近调节冲突（或预期的“虚拟焦点”与显示器的实际焦平面分离）是许多这些心理物理问题的原因。PI在这个项目中的目标是通过利用硅纳米光子学的进步来开发一种自然的VR和AR多焦点显示原型，以及多焦点和自动多视角显示所需的视觉计算算法来解决这种不匹配问题。PI还将设计并开展用户研究，以验证新技术（显示和算法）。项目成果不仅会直接影响计算机图形学和可视化领域，还会影响一系列其他领域。大规模流体流动模拟的多焦点可调节精确立体可视化将代表科学和工程领域的突破，从人造动脉瓣膜的设计到飞行器，通过蛋白质对接的合理药物设计过程，以及可以利用实时MRI技术和其他高维医学成像数据的外科进步。同样重要的是，这项新技术的潜在教育效益，它将释放VR和AR的力量，让年轻学生沉浸在他们通常无法接触的世界中，提供兴奋、参与和广度和深度的背景，这在传统课堂环境中几乎是不可能实现的。该项目的核心在于设计和实现一种新型纳米光子相控阵芯片，该芯片将能够使用大规模相控阵产生任意辐射模式，这将扩展相控阵的功能，超越传统的波束聚焦和转向，通信和雷达，并将在图像处理，3D交互式全息和虚拟现实方面开辟新的机会。该项目汇集了在3D计算机图形学和科学可视化、纳米光子学和等离子体学（用于亚波长限制和控制光）、微电子学、集成电路和微结构科学和技术方面具有重要专业知识的合作者。该研究将涉及三个重点：(a)慢光，以显着减少纳米光子芯片的尺寸和功率要求；(b)将芯片的电子和光学组件分开，以更好地优化每个组件；(c)开发有效的算法，通过用户研究，在傅里叶域中呈现和验证多焦点3D图形。多焦显示器的设计是风险最高、收益最高的工作组件；该团队已经制定了应急计划，以便在必要时继续开发多焦和自动多镜显示算法，并利用现有的多焦平面显示器进行用户研究。

The current explosion of virtual and augmented reality (VR and AR) technologies is the subject of much academic, industrial and popular enthusiasm. Unfortunately, these technologies have also been shown to induce a host of serious user complaints, including headaches, nausea and blurred vision. This is particularly problematic for 3D displays and content, and the National Academy of Engineering has identified "enhancing virtual reality interfaces" as a grand challenge for the 21st century. Studies have suggested that the vergence-accommodation conflict (or the decoupling of the intended "virtual focus" from the actual focal plane of a display) is the cause of many of these psychophysical problems. The PI's goal in this project is to address this mismatch by utilizing advances in silicon nanophotonics to develop a prototype natural-to-the-senses VR and AR multifocal display along with the requisite visual computing algorithms for multifocal and automultiscopic displays. The PI will also design and carry out user studies to validate the new technology (display and algorithms). Project outcomes will not only directly impact the fields of computer graphics and visualization, but an array of other fields as well. Multifocal accommodative-accurate stereo visualization of large-scale fluid flow simulations would represent a breakthrough in science and engineering generally, in areas ranging from the design of artificial arterial valves to aerial vehicles, the rational drug design process through protein docking, and in surgical advances that could take advantage of real-time MRI technologies and other high-dimensional medical imaging data. No less important are the potential educational benefits of the new technology, which would unleash the power of VR and AR to immerse young students in worlds to which they would normally not have access, providing excitement, engagement and a breadth and depth of context that is nearly impossible to achieve in a traditional classroom setting.The heart of this project lies in the design and implementation of a novel nanophotonic phased-array chip that will enable the generation of arbitrary radiation patterns with large-scale phased arrays, which would extend the functionality of phased arrays beyond conventional beam focusing and steering, communication and radar, and would open up new opportunities in image processing, 3D interactive holography, and virtual reality. This project brings together collaborators with significant expertise in 3D computer graphics and scientific visualization, in nanophotonics and plasmonics for the sub-wavelength confinement and steering of light, and in microelectronics, integrated circuits, and microstructure science and technology. The research will involve three thrusts: (a) slow light to significantly reduce the size and power requirements for the nanophotonics chip; (b) separate electronic and optical components of the chip to better optimize each; and (c) develop efficient algorithms to render and validate, through user studies, multifocal 3D graphics in the Fourier domain. The design of the multifocal display is the highest-risk and highest-gain component of the work; the team has contingency plans in place to proceed if necessary with developing multifocal and automultiscopic display algorithms, and for conducting user studies, employing existing multiple-focal-plane displays.