Collaborative Research: CNS Core: HCC: Small: Enabling Efficient Computer Systems for Augmented and Virtual Reality: A Perception-Guided Approach

这项研究项目设计了软硬件协作机制，以弥合人类视觉感知和增强/虚拟现实(AR/VR)系统设计之间的差距，并最终实现数量级的效率提升。要实现这一目标，首先需要开发计算模型来定量捕获人类的视觉感知，然后在整个系统堆栈中研究利用感知模型进行有效处理的新技术。该项目的智力价值在于追求与利用人类感知的独特特征建立高效AR/VR系统有关的三类技术：1)确定同时影响观看体验和系统效率的视觉刺激的关键方面，并通过广泛的心理物理实验研究轻量级建模策略；2)研究整个应用程序管道(从算法和系统到显示器)的使能技术，利用感知模型来提高系统效率；3)开发一种数据驱动的、基于学习的方法来调整整个系统，同时满足感知和效率要求。研究议程由教育/推广议程补充。PIS将(1)为罗切斯特和纽约的高中生提供暑期研究和编程机会，(2)在罗切斯特大学和纽约大学引入关于视觉感知及其与计算机系统设计的联系的新课程/模块，(3)与行业合作伙伴接触和合作，探索技术转移的可能性，以及(4)为本科生提供在新兴应用领域和硬件加速方面的实践经验的包容性机会。总体而言，综合研究和教育活动将引领新产品和想法的激增，进而解决医疗保健、教育、远程通信和交互、专业培训和工业设计等领域的关键技术和社会问题。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research project designs software-hardware collaborative mechanisms to bridge the gap between human visual perception and augmented/virtual reality (AR/VR) systems design and ultimately deliver an order of magnitude efficiency gains. Achieving this goal requires first developing computational models to quantitatively capture human visual perception, and then investigating new techniques across the systems stack that leverage the perceptual models for efficient processing. The intellectual merit of the project is the pursuit of three classes of techniques pertaining to exploiting unique characteristics of human perception to build efficient AR/VR systems: 1) identify key aspects of visual stimuli that simultaneously influence viewing experience and system efficiency and investigate lightweight modeling strategies through extensive psychophysical experiments, 2) investigate enabling techniques across the entire application pipeline (from algorithms and systems to the display) that leverage the perceptual models for improving system efficiency, and 3) develop a data-driven, learning-based approach to tune the entire system while satisfying perceptual and efficiency requirements.The research agenda is complemented by an educational/outreach agenda. The PIs will (1) offer summer research and programming opportunities to high school students in Rochester and New York, (2) introduce new courses/modules in both University of Rochester and NYU on visual perception and its connection to computer systems design, (3) reach out and collaboration with industry partners to explore technology transfer possibilities, and (4) offer undergraduate students inclusive opportunities for hands-on experience in emerging application domains and hardware acceleration. Broadly, the integrated research and education activities will lead the way to an explosion of new products and ideas, which in turn addresses key technological and societal issues in areas such as healthcare, education, remote communication and interaction, professional training, and industrial design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.