HCC: Small: Development of an Eyeglass-Style Compact Eye-Tracked Near-Eye Display Using Freeform Optical Technology

一种集成的眼球追踪头戴式显示器（ET-HMD）能够显示立体虚拟图像，同时还能跟踪用户的视线方向，这将有利于基础科学研究和许多新兴应用。然而，尽管独立的头戴式显示器和眼动追踪技术取得了重大进展，并且商业化，但一种符合太阳镜外形的便携、轻便、精确和强大的系统仍然难以实现。PI在这个项目中的目标是开发一种全新的光学技术，通过克服迄今为止应用于ET-HMD设计的传统方法的局限性，使实现这一梦想成为可能。在以往设计ET-HMD系统的工作中，大多数将HMD和眼动追踪路径的光学系统分开处理，并使用旋转对称的光学表面。相比之下，PI的方法是利用自由曲面光学技术与创新的光学方案相结合，独特地将显示光学与眼睛成像光学相结合。为此，她将研究设计自由曲面ET-HMD系统的挑战，并为所需的高性能光学系统开发适当的设计方法和优化策略。她将实现一个完全集成的便携式原型系统，并开发用于评估显示和眼球追踪性能的校准和评估方法。作为新技术的试验应用，PI将评估采用ET-HMD显示器增强ALS（肌萎缩侧索硬化症）或导致语言丧失的类似神经系统疾病患者交流的可行性。项目成果将包括首个轻便便携式ET-HMD显示器原型，其外形尺寸接近太阳镜。更广泛的影响：该项目产生的新技术将在许多领域产生关键影响。它将为移动通信、可穿戴计算和便携式娱乐创造一个革命性的、有趣的平台。它将为与视觉和人为因素相关的研究提供一个新的工具，其中眼球运动为理解人类在3D空间中的感知和特定任务的有效性提供了一个合理的度量。它将为本体感觉残疾或因手脚被占用而造成情境障碍的人提供辅助和替代的人机界面。毫无疑问，它将刺激一系列新的、令人兴奋的应用程序的开发。这项研究还将为在本科和研究生两级培训下一代跨学科科学家和工程师提供一种工具。

An integrated eye-tracking head-mounted display (ET-HMD) able to display stereoscopic virtual images while also tracking the direction of the user's gaze would benefit both fundamental scientific research and a host of emerging applications. Yet despite significant advances in and commercial availability of stand-alone HMD and eye-tracking technologies, a portable, lightweight, accurate and robust system that conforms to the form factor of a pair of sunglasses remains elusive. The PI's goal in this project is to develop a fundamentally new optical technology that will make it possible to realize this dream by overcoming the limitations of the conventional methods that have been applied to ET-HMD designs to date. In most previous work on designing ET-HMD systems, the optical systems for the HMD and eye-tracking paths were treated separately, and rotationally symmetric optical surfaces were used. In contrast, the PI's approach is to utilize freeform optical technology in combination with an innovative optical scheme that uniquely combines the display optics with the eye imaging optics. To these ends, she will investigate the challenges of designing the freeform ET-HMD system and develop appropriate design methods along with an optimization strategy for the required high performance optical system. She will implement a fully integrated, portable prototype system, and develop calibration and assessment methods for evaluating both the display and eye-tracking performance. As a testbed application for the new technology, the PI will evaluate the feasibility of adopting ET-HMD displays to augment communication by patients suffering from ALS (amyotrophic lateral sclerosis) or similar neurological disease that causes loss of speech. Project outcomes will include the first lightweight and portable ET-HMD display prototype that has a form factor close to that of sunglasses.Broader Impacts: The new technology resulting from this project will have critical impacts in many fields. It will create a revolutionary and intriguing platform for mobile communication, wearable computing, and portable entertainment. It will provide a new tool for research related to vision and human factors, where eye movements provide a sensible metric for understanding human perception in 3D space and effectiveness at specific tasks. It will afford augmentative and alternative human-computer interfaces for people with proprioceptive disabilities or with situational impairments due to having their hands and feet occupied. And it will undoubtedly spur development of a host of new and thrilling applications. The research will also provide a vehicle for training the next generation of interdisciplinary scientists and engineers, at both the undergraduate and graduate levels.