CAREER: Rhythmic Pixel Region Interface Systems for Efficient, Performant, and Precise Augmented Reality

增强现实（AR）系统允许虚拟对象通过使用AR眼镜或通过移动终端的相机/屏幕在视觉上覆盖在物理空间中。这为教育提供了丰富的交互式体验，为劳动力培训提供了远程指导，为自己动手建造房屋提供了身临其境的手册，建筑行业监控，为消费者购买提供了室内设计设想，以及娱乐游戏，仅举几例。 然而，当前的AR系统由于与具有高图像帧分辨率和高帧速率的视觉计算相关联的高数据速率而遭受高能耗和有限的性能。该项目旨在降低视觉计算的传感数据速率，使更紧凑的增强现实设备具有更小的电池尺寸，以及更高精度地将虚拟对象放置在物理空间中。为此，该项目涉及重新设计视觉计算硬件和软件系统，围绕处理和产生具有不同空间分辨率和时间间隔的图像像素区域，选择性地由增强现实软件需求指导。为此，该项目将引入新的方法来设计和表征自适应传感架构。这将包括硬件架构和软件框架模式的调查，以提供传感器操作的应用程序控制。此外，该项目还将提高K-12学生以及本科生和研究生在科学、技术、工程和数学方面的参与度，通过开发和演示一种教育增强现实体验来吸引学生，该体验描述了在计算机视觉和增强现实应用背景下图像传感器系统的操作。该项目研究产生了系统软件、应用程序和实验测试台。这些将包括源代码文件和输入数据，以测试系统，包括图像和视频。该项目提供了如何重现实验的所有步骤的详细说明。该项目还提供实验结果来描述功耗、性能、任务精度和其他表征数据。这些都托管在一个软件库网站上，并通过一个可访问的网页链接：http://meteor.ame.asu.edu/rhythmicpixelregions。项目结束后5年，或项目发布后5年，以较晚者为准。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

An augmented reality (AR) system allows for virtual objects to be overlaid visually in physical spaces through the use of AR glasses or through the camera/screen of a mobile device. This provides rich interactive experiences for education, remote guidance for workforce training, immersive manuals for do-it-yourself home construction, construction industry monitoring, interior design envisioning for consumer purchases, and entertaining games, to name just a few areas. However, current AR systems suffer from high energy consumption and limited performance due to the high data rates associated with visual computing with high image frame resolutions and high frame rates. The proposed project aims to reduce the sensing data rate of visual computing, enabling more compact augmented reality devices with smaller battery sizes, and higher precision placement of virtual objects in the physical spaces. To this end, the project involves a redesign of the visual computing hardware and software systems around processing and producing image pixel regions with varying spatial resolutions and temporal intervals, selectively guided by the augmented reality software needs. To this end, the project will introduce new methods to design and characterize adaptive sensing architectures. This will include the investigation of hardware architecture and software framework patterns to provide application control of sensor operation. In addition, the proposed project advances K-12 student engagement and undergraduate and graduate student engagement in science, technology, engineering and math, engaging students through the development and demonstration of an educational augmented reality experience describing image sensor system operation in the context of computer vision and augmented reality applications.The project research produces system software, applications, and experimental testbenches. These will include source code files and input data to test the system, including images and videos. The project provides detailed instructions on how to reproduce all steps of the experiments. The project also provides experimental results to describe power consumption, performance, task accuracy, and other characterization data. These are hosted on a software repository site, and linked through an accessible webpage: http://meteor.ame.asu.edu/rhythmicpixelregions. The repository and the webpage will remain active for 5 years after the end of the project, or 5 years after publication, whichever is later.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.