III:Small: Data Exploration Framework for Augmented Reality

随着计算机视觉的最新进展和功能强大的硬件的广泛可用性，增强现实已经成为一种消费级技术。它开创了一种使用基于摄像头的设备进行数字交互的新模式，呈现出与现实世界交互的新方式。与此同时，另一个新兴趋势是现实世界中数据探索的盛行。探索真实世界数据的一个挑战是，这些数据往往是孤立的，缺乏任何上下文信息。此外，此类数据通常采用临时格式，因此很难与数字数据结合使用。在这些趋势的基础上，该项目旨在实现增强现实（AR）驱动的发现和对特定现实世界数据的决策。这种使用基于ar的交互进行数据探索和丰富的方法将在各种领域开辟可能性，包括教育、医疗保健、架构设计和紧急服务。增强现实扩大了那些无法使用键盘等传统计算界面的群体的参与范围，解决了识字和运动能力方面的挑战。该项目将开发一个原则性框架，用于使用AR探索和理解现实世界的数据。该项目的主要目标如下。作为第一步，该项目将设计一种新的语法和可视化编码，允许在AR空间中查询和分层结构化数据结果。其次，它将设计方法，通过使用基于云的数据存储，不仅考虑光学字符识别，而且考虑视觉特征和上下文，用丰富的模式信息来丰富现实世界的图像和视频流。第三，该项目将开发一个数据存储和查询处理后端，该后端专门针对高度交互的、特定于ar的工作负载进行了优化。这项研究的结果不仅将促进对构建AR数据探索系统的理解，而且还将有助于降低软件开发人员构建数据丰富的AR应用程序的进入门槛。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With recent advances in computer vision and widespread availability of capable hardware, augmented reality has become a consumer-grade technology. It has enabled a new paradigm of digital interaction using camera-based devices, presenting novel ways to interact with the real world. Simultaneously, another emerging trend is the prevalence of data exploration in the real world. One challenge in exploring real-world data is that it is often found in isolation, lacking any contextual information. Further, such data is commonly found in ad-hoc formats, making it hard to combine with digital data. Building on these trends, this project aims to enable augmented reality (AR)-driven discovery and decision making over ad-hoc real-world data. This approach towards data exploration and enrichment using AR-based interaction will open up possibilities in a wide variety of domains, including education, healthcare, architecture design, and emergency services. AR broadens the scope of participation for groups who would otherwise be unable to use traditional computing interfaces such as keyboards, addressing challenges of literacy and motor abilities.This project will develop a principled framework for exploration and understanding of real-world data using AR. The primary goals of this project are as follows. As a first step, the project will design a new grammar and visual encoding that allows for both querying and layering of structured data results in AR space. Secondly, it will devise methods to enrich real-world images and video streams with schema-rich information using cloud-based data stores by considering not just optical character recognition, but also visual features and context. Third, the project will develop a data storage and query processing backend that is optimized specifically for highly interactive, AR-specific workloads. The outcomes of this research will not only advance the understanding of building data exploration systems for AR, but it will additionally help reduce the barrier to entry for software developers to build data-rich AR applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.