Context-Driven Salience for Querying and Visualizing Immersive 3D Content

用于查询和可视化沉浸式 3D 内容的上下文驱动显着性

• 批准号: RGPIN-2016-05398
• 负责人: Kyan, Matthew
• 金额: $ 2.26万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=709672
• 关键词: Context; Driven; Salience; Querying; Visualizing

Steady advances in the fields of surveying and remote sensing, 3D modelling acquisition and reconstruction, and digital-content delivery, offer new opportunities for researchers, students, and the general public to generate or obtain rich 2D and 3D visual data pertaining to various aspects of the world we live in. To date, much progress has been made in the sensing and acquisition of rich visual, spatially associated data (i.e. 3D scans, models and associated source imagery), their efficient representation, visualization and even physical reproduction (e.g. 3D printing). However, in areas such as cultural heritage or in establishing a digital record that can support general scientific enquiry, there is a pressing need for tools and methodologies that support the formation of queries to extract, analyse and dissect. Furthermore, due to the size and complexity of large-scale heterogeneous visual data, tools must also be able to facilitate human investigation in a way that can effectively support cognition (without overburdening the user). Recent developments in tangible user interfaces (TUI) and embodied cognition in particular, have identified a link between the use of motor skills and the enhanced cognition that it stimulates. Extending visual data mining paradigms to include such interfaces thus seems logical. To this end, we aim to investigate such tools on several fronts: 1) through the detection of novelty in the data based on visual salience, which will be considered in terms of both 2D image, and 3D structural features; and 2) through the unsupervised structuring and organization of detected novelty across different spatial locations, and across different sources of heterogeneous visual data, in order to establish contextual descriptions indicating candidate locations and details in the data that can be targeted for investigation, comparison and analysis. Finally, 3) we will investigate the integration of these principles within tangible user interfaces and tangible query construction for visualization and exploration of rich visual data. In the proposed program of research, we will first advance the state-of-the-art in 2D/3D visual salience as suitable descriptors for identifying informative details of interest. We will develop methodologies for systematic compartmentalization and characterization of visual novelty, and explore frameworks in which such novelty can be made readily accessible to the user in support of effective and targeted inquiry.

测量和遥感、3D建模获取和重建以及数字内容交付领域的稳步发展为研究人员、学生和普通公众提供了新的机会，以生成或获取与我们生活的世界各个方面有关的丰富的2D和3D可视数据。 到目前为止，在感知和获取丰富的视觉、空间相关数据(即3D扫描、模型和相关的源图像)及其高效表示、可视化甚至物理复制(例如3D打印)方面取得了很大进展。然而，在文化遗产或建立能够支持一般科学查询的数字记录等领域，迫切需要支持形成查询的工具和方法，以提取、分析和剖析查询。 此外，由于大规模异质视觉数据的大小和复杂性，工具还必须能够以一种能够有效支持认知的方式促进人类调查(而不会使用户负担过重)。有形用户界面(TUI)的最新发展，特别是具体化认知，已经确定了运动技能的使用与其刺激的增强认知之间的联系。因此，将可视化数据挖掘范例扩展到包括这样的界面似乎是合乎逻辑的。 为此，我们的目标是在几个方面研究这类工具：1)通过基于视觉突出度检测数据中的新颖性，这将从2D图像和3D结构特征两个方面进行考虑；以及2)通过跨不同空间位置和不同来源的不同视觉数据的无监督结构和组织检测到的新颖性，以便建立上下文描述，以指示数据中的候选位置和细节，可以作为调查、比较和分析的目标。最后，我们将研究这些原则在有形用户界面和有形查询构造中的集成，以可视化和探索丰富的可视数据。 在拟议的研究计划中，我们将首先推进2D/3D视觉显著的最新进展，作为识别感兴趣的信息细节的合适描述符。我们将制定对视觉新颖性进行系统划分和定性的方法，并探索可使用户容易获得这种新颖性的框架，以支持有效和有针对性的查询。