SHF: Medium: Collaborative Research: Enhancing Mobile VR/AR User Experience: An Integrated Architecture-System Approach

虚拟现实(VR)和增强现实(AR)正在经历快速增长，并在计算硬件显著提高的性能和图形显示技术革命的推动下首次进入主流市场。VR和AR都越来越多地被智能手机采用，为移动用户提供丰富的体验。然而，移动设备中有限的硬件资源几乎不能支持VR/AR技术对巨大资源的需求和对良好用户体验的需求。在智能手机上启用VR/AR应用程序时，有几个关键挑战会显著降低用户体验。例如，当VR用户稍微移动她/他的眼睛时，计算机无法及时提供与新视图相对应的图像。这种延迟会导致运动异常和令人不满意的用户体验。移动VR/AR面临的其他挑战包括电池寿命有限和散热增加。该项目的研究人员正在利用VR/AR的独特功能，探索一种协同架构-系统方案，以解决上述性能、电池寿命和温度挑战，从而增强移动VR/AR用户体验。该项目将为下一代移动平台打开大门，提供高质量、低功耗的VR/AR服务，以满足移动用户的需求。该项目还将通过吸引代表不足的群体、扩展到高中生、开发物联网课程以及传播用于教育和培训的研究基础设施来贡献社会。该项目包含四个目标，包括(1)近似计算，以消除移动AR/VR中不必要的工作量，在不牺牲用户感知的图像质量和输出精度的情况下，在性能、功率和热量上实现三倍效益；(2)新兴技术(例如，内存中处理、多芯片模块(MCM)-GPU)使移动VR/AR能够进行性能/功率优化；(3)利用动态热能收集来冷却热点，延长电池寿命和提高性能；以及(4)整合关键研究创新和跨技术优化，以最大限度地提高性能/功率/热增强。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual reality (VR) and augmented reality (AR) are now experiencing rapid growth and debuting into mainstream markets driven by the significant performance improvement of computing hardware and the revolution of graphics and display technologies. Both VR and AR are increasingly adopted in smartphones for providing rich experience to mobile users. However, the limited hardware resources in mobile devices can barely support the tremendous resource requirements by VR/AR technology and the demand for a good user experience. There are several key challenges that dramatically degrade user experience when enabling VR/AR apps on smartphones. For example, when the VR user shifts her/his eyes slightly, the computer is unable to provide an image that corresponds to the new view in a timely manner. This delay causes motion anomalies and unsatisfying user experience. Other challenges for mobile VR/AR are limited battery life and increased heat dissipation. The investigators on this project are leveraging the unique features of VR/AR and exploring a synergetic architecture-system program to tackle the above performance, battery life, and thermal challenges, thus, enhancing mobile VR/AR user experience. This project will open the door for next generation mobile platforms that provide high-quality low-power VR/AR services to satisfy mobile users. This project will also contribute to society through engaging under-represented groups, and outreach to high-school students, curriculum development on Internet of Things, and disseminating research infrastructure for education and training.This project contains four objectives, including (1) approximate computing to eliminate unnecessary workloads in mobile AR/VR to achieve threefold benefits on performance, power, and thermal without sacrificing the user perceived image quality and output accuracy; (2) emerging technologies (e.g., processing-in-memory, multiple chip module (MCM)-GPU) enabled mobile VR/AR for performance/power optimization; (3) exploiting dynamic thermal energy harvesting to cool hotspots, prolong battery life and improve performance; and (4) integration of the key research innovations and cross-technology optimizations to maximize the performance/power/thermal enhancement.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.