Collaborative Research: CNS Core: HCC: Small: Enabling Efficient Computer Systems for Augmented and Virtual Reality: A Perception-Guided Approach

合作研究：CNS 核心：HCC：小型：为增强现实和虚拟现实启用高效计算机系统：感知引导方法

• 批准号: 2225861
• 负责人: Qi Sun
• 金额: $ 30万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-11-01 至 2025-10-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2225861&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; HCC

This research project designs software-hardware collaborative mechanisms to bridge the gap between human visual perception and augmented/virtual reality (AR/VR) systems design and ultimately deliver an order of magnitude efficiency gains. Achieving this goal requires first developing computational models to quantitatively capture human visual perception, and then investigating new techniques across the systems stack that leverage the perceptual models for efficient processing. The intellectual merit of the project is the pursuit of three classes of techniques pertaining to exploiting unique characteristics of human perception to build efficient AR/VR systems: 1) identify key aspects of visual stimuli that simultaneously influence viewing experience and system efficiency and investigate lightweight modeling strategies through extensive psychophysical experiments, 2) investigate enabling techniques across the entire application pipeline (from algorithms and systems to the display) that leverage the perceptual models for improving system efficiency, and 3) develop a data-driven, learning-based approach to tune the entire system while satisfying perceptual and efficiency requirements.The research agenda is complemented by an educational/outreach agenda. The PIs will (1) offer summer research and programming opportunities to high school students in Rochester and New York, (2) introduce new courses/modules in both University of Rochester and NYU on visual perception and its connection to computer systems design, (3) reach out and collaboration with industry partners to explore technology transfer possibilities, and (4) offer undergraduate students inclusive opportunities for hands-on experience in emerging application domains and hardware acceleration. Broadly, the integrated research and education activities will lead the way to an explosion of new products and ideas, which in turn addresses key technological and societal issues in areas such as healthcare, education, remote communication and interaction, professional training, and industrial design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目设计了软件-硬件协作机制，以弥合人类视觉感知和增强/虚拟现实（AR/VR）系统设计之间的差距，并最终实现一个数量级的效率增益。要实现这一目标，首先需要开发计算模型来定量捕获人类视觉感知，然后研究跨系统堆栈的新技术，利用感知模型进行有效处理。该项目的智力价值在于追求三类技术，这些技术与利用人类感知的独特特征来构建高效的AR/VR系统有关：1）识别同时影响观看体验和系统效率的视觉刺激的关键方面，并通过广泛的心理物理实验研究轻量级建模策略，2）研究整个应用程序管道中的启用技术（从算法和系统到显示器），其利用感知模型来提高系统效率，以及3）开发数据驱动的，以学习为基础的方法来调整整个系统，同时满足感知和效率要求。研究议程由教育/推广议程补充。PI将（1）为罗切斯特和纽约的高中生提供暑期研究和编程机会，（2）在罗切斯特大学和纽约大学引入视觉感知及其与计算机系统设计的联系的新课程/模块，（3）与行业合作伙伴接触和合作，探索技术转让的可能性，以及（4）为本科生提供在新兴应用领域和硬件加速方面获得实践经验的机会。从广义上讲，综合研究和教育活动将引领新产品和新想法的爆炸式增长，从而解决医疗保健，教育，远程通信和互动，专业培训，该奖项反映了NSF的法定使命，并被认为是值得通过评估使用基金会的智力价值和更广泛的支持。影响审查标准。

项目成果

Reconstructing room scales with a single sound for augmented reality displays

使用单一声音重建房间尺度以进行增强现实显示

• DOI: 10.1080/15980316.2022.2145377
• 发表时间: 2023
• 期刊: Journal of Information Display
• 影响因子: 3.7
• 作者: Liang, Benjamin S.;Liang, Andrew S.;Roman, Iran;Weiss, Tomer;Duinkharjav, Budmonde;Bello, Juan Pablo;Sun, Qi
• 通讯作者: Sun, Qi

Force-Aware Interface via Electromyography for Natural VR/AR Interaction

• DOI: 10.1145/3550454.3555461
• 发表时间: 2022-10
• 期刊: ACM Transactions on Graphics (TOG)
• 作者: Yunxiang Zhang;Benjamin Liang;Boyuan Chen-;P. Torrens;S. F. Atashzar;Dahua Lin;Qinghong Sun

Imperceptible Color Modulation for Power Saving in VR/AR

难以察觉的色彩调制，可在 VR/AR 中实现省电

• DOI: 10.1145/3588037.3595388
• 发表时间: 2023
• 期刊: ACM SIGGRAPH 2023 Emerging Technologies
• 作者: Chen, Kenneth;Duinkharjav, Budmonde;Ujjainkar, Nisarg;Shahan, Ethan;Tyagi, Abhishek;He, Jiayi;Zhu, Yuhao;Sun, Qi
• 通讯作者: Sun, Qi