CAREER: Rhythmic Pixel Region Interface Systems for Efficient, Performant, and Precise Augmented Reality

职业：用于高效、高性能和精确增强现实的节奏像素区域接口系统

• 批准号: 1942844
• 负责人: Robert LiKamWa
• 金额: $ 45万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-04-01 至 2025-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942844&HistoricalAwards=false
• 关键词: CAREER; Rhythmic; Pixel; Region; Interface

An augmented reality (AR) system allows for virtual objects to be overlaid visually in physical spaces through the use of AR glasses or through the camera/screen of a mobile device. This provides rich interactive experiences for education, remote guidance for workforce training, immersive manuals for do-it-yourself home construction, construction industry monitoring, interior design envisioning for consumer purchases, and entertaining games, to name just a few areas. However, current AR systems suffer from high energy consumption and limited performance due to the high data rates associated with visual computing with high image frame resolutions and high frame rates. The proposed project aims to reduce the sensing data rate of visual computing, enabling more compact augmented reality devices with smaller battery sizes, and higher precision placement of virtual objects in the physical spaces. To this end, the project involves a redesign of the visual computing hardware and software systems around processing and producing image pixel regions with varying spatial resolutions and temporal intervals, selectively guided by the augmented reality software needs. To this end, the project will introduce new methods to design and characterize adaptive sensing architectures. This will include the investigation of hardware architecture and software framework patterns to provide application control of sensor operation. In addition, the proposed project advances K-12 student engagement and undergraduate and graduate student engagement in science, technology, engineering and math, engaging students through the development and demonstration of an educational augmented reality experience describing image sensor system operation in the context of computer vision and augmented reality applications.The project research produces system software, applications, and experimental testbenches. These will include source code files and input data to test the system, including images and videos. The project provides detailed instructions on how to reproduce all steps of the experiments. The project also provides experimental results to describe power consumption, performance, task accuracy, and other characterization data. These are hosted on a software repository site, and linked through an accessible webpage: http://meteor.ame.asu.edu/rhythmicpixelregions. The repository and the webpage will remain active for 5 years after the end of the project, or 5 years after publication, whichever is later.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

增强现实（AR）系统允许通过使用AR眼镜或通过移动设备的摄像头/屏幕在物理空间中视觉上覆盖虚拟对象。这为教育提供了丰富的互动体验，为劳动力培训提供了远程指导，为自己动手的家庭建筑提供了沉浸式手册，建筑行业监控，为消费者购买提供了室内设计设想，以及娱乐游戏，仅举几个领域。然而，当前的AR系统由于与高图像帧分辨率和高帧率的视觉计算相关的高数据速率而遭受高能耗和有限的性能。拟议的项目旨在降低视觉计算的传感数据速率，使更紧凑的增强现实设备具有更小的电池尺寸，并在物理空间中更高精度地放置虚拟物体。为此，该项目涉及重新设计视觉计算硬件和软件系统，围绕处理和产生具有不同空间分辨率和时间间隔的图像像素区域，有选择地由增强现实软件需求指导。为此，该项目将引入新的方法来设计和表征自适应传感体系结构。这将包括硬件架构和软件框架模式的研究，以提供传感器操作的应用控制。此外，拟议的项目促进了K-12学生以及本科生和研究生在科学、技术、工程和数学方面的参与，通过开发和演示教育增强现实体验来吸引学生，该体验描述了计算机视觉和增强现实应用背景下图像传感器系统的操作。该项目研究产生了系统软件、应用程序和实验测试平台。这些将包括源代码文件和用于测试系统的输入数据，包括图像和视频。该项目提供了如何重现实验所有步骤的详细说明。该项目还提供了描述功耗、性能、任务准确性和其他特征数据的实验结果。它们托管在一个软件存储库站点上，并通过一个可访问的网页链接：http://meteor.ame.asu.edu/rhythmicpixelregions。存储库和网页将在项目结束后的5年内保持活跃，或在发布后的5年内保持活跃，以较晚者为准。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Rhythmic pixel regions: multi-resolution visual sensing system towards high-precision visual computing at low power

• DOI: 10.1145/3445814.3446737
• 发表时间: 2021-04
• 期刊: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems
• 作者: Venkatesh Kodukula;Alexander Shearer;Van Nguyen;Srinivas Lingutla;Yifei Liu;R. Likamwa