SHF: Medium: Collaborative Research: Enhancing Mobile VR/AR User Experience: An Integrated Architecture-System Approach

虚拟现实（VR）和增强现实（AR）现在正在经历快速增长，并在计算硬件的显着性能改进以及图形和显示技术革命的推动下进入主流市场。智能手机越来越多地采用VR和AR，为移动的用户提供丰富的体验。然而，移动的设备中有限的硬件资源难以支持VR/AR技术的巨大资源需求和对良好用户体验的需求。在智能手机上启用VR/AR应用程序时，有几个关键挑战会显著降低用户体验。例如，当VR用户稍微移动她/他的眼睛时，计算机无法及时提供与新视图对应的图像。该延迟导致运动异常和不令人满意的用户体验。移动的VR/AR的其他挑战是有限的电池寿命和增加的散热。该项目的研究人员正在利用VR/AR的独特功能，并探索协同架构系统计划，以解决上述性能，电池寿命和热挑战，从而增强移动的VR/AR用户体验。该项目将为下一代移动的平台打开大门，提供高质量的低功耗VR/AR服务，以满足移动的用户。本项目还将通过面向未被充分代表的群体、面向高中生、开发物联网相关课程、普及教育培训的研究基础设施等方式，为社会做出贡献。本项目包含四个目标，包括（1）近似计算，消除移动的AR/VR中不必要的工作量，实现性能、功耗、并且不牺牲用户感知的图像质量和输出精度;（2）新兴技术（例如，内存处理、多芯片模块（MCM）-GPU）支持移动的VR/AR，实现性能/功耗优化;（3）利用动态热能收集来冷却热点，延长电池寿命并提高性能;以及（4）集成关键研究创新和交叉技术优化，以最大限度地提高性能/功率/热增强。该奖项反映了NSF的法定使命，并且通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual reality (VR) and augmented reality (AR) are now experiencing rapid growth and debuting into mainstream markets driven by the significant performance improvement of computing hardware and the revolution of graphics and display technologies. Both VR and AR are increasingly adopted in smartphones for providing rich experience to mobile users. However, the limited hardware resources in mobile devices can barely support the tremendous resource requirements by VR/AR technology and the demand for a good user experience. There are several key challenges that dramatically degrade user experience when enabling VR/AR apps on smartphones. For example, when the VR user shifts her/his eyes slightly, the computer is unable to provide an image that corresponds to the new view in a timely manner. This delay causes motion anomalies and unsatisfying user experience. Other challenges for mobile VR/AR are limited battery life and increased heat dissipation. The investigators on this project are leveraging the unique features of VR/AR and exploring a synergetic architecture-system program to tackle the above performance, battery life, and thermal challenges, thus, enhancing mobile VR/AR user experience. This project will open the door for next generation mobile platforms that provide high-quality low-power VR/AR services to satisfy mobile users. This project will also contribute to society through engaging under-represented groups, and outreach to high-school students, curriculum development on Internet of Things, and disseminating research infrastructure for education and training.This project contains four objectives, including (1) approximate computing to eliminate unnecessary workloads in mobile AR/VR to achieve threefold benefits on performance, power, and thermal without sacrificing the user perceived image quality and output accuracy; (2) emerging technologies (e.g., processing-in-memory, multiple chip module (MCM)-GPU) enabled mobile VR/AR for performance/power optimization; (3) exploiting dynamic thermal energy harvesting to cool hotspots, prolong battery life and improve performance; and (4) integration of the key research innovations and cross-technology optimizations to maximize the performance/power/thermal enhancement.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.