Context-Driven Salience for Querying and Visualizing Immersive 3D Content

在测量和遥感、3D建模获取和重建以及数字内容交付领域的稳步发展，为研究人员、学生和公众提供了新的机会，可以生成或获取与我们生活的世界各个方面有关的丰富的2D和3D视觉数据。******迄今为止，在感知和获取丰富的视觉、空间相关数据（即3D扫描、模型和相关源图像）、它们的有效表示、可视化甚至物理复制（例如3D打印）方面取得了很大进展。然而，在文化遗产或建立可支持一般科学探究的数字记录等领域，迫切需要工具和方法来支持查询的形成，以便提取、分析和剖析。******此外，由于大规模异构视觉数据的大小和复杂性，工具还必须能够以一种能够有效支持认知的方式促进人类调查（而不会使用户负担过重）。最近在有形用户界面（TUI），特别是具身认知方面的发展，已经确定了运动技能的使用与它所刺激的增强认知之间的联系。因此，扩展可视化数据挖掘范例以包含这样的接口似乎是合乎逻辑的。******为此，我们的目标是在几个方面研究这些工具：1)通过基于视觉显著性的数据新颖性检测，这将在2D图像和3D结构特征方面进行考虑；2)通过对不同空间位置、不同来源的异质视觉数据中检测到的新颖性进行无监督的结构化和组织，以建立表明候选位置和数据细节的上下文描述，从而有针对性地进行调查、比较和分析。最后，我们将研究这些原则在有形用户界面和有形查询构建中的集成，以实现可视化和探索丰富的可视化数据。******在拟议的研究计划中，我们将首先推进最先进的2D/3D视觉显著性，作为识别感兴趣的信息细节的合适描述符。我们将开发系统划分和表征视觉新颖性的方法，并探索框架，使用户可以很容易地获得这种新颖性，以支持有效和有针对性的查询。

Steady advances in the fields of surveying and remote sensing, 3D modelling acquisition and reconstruction, and digital-content delivery, offer new opportunities for researchers, students, and the general public to generate or obtain rich 2D and 3D visual data pertaining to various aspects of the world we live in. ******To date, much progress has been made in the sensing and acquisition of rich visual, spatially associated data (i.e. 3D scans, models and associated source imagery), their efficient representation, visualization and even physical reproduction (e.g. 3D printing). However, in areas such as cultural heritage or in establishing a digital record that can support general scientific enquiry, there is a pressing need for tools and methodologies that support the formation of queries to extract, analyse and dissect. ******Furthermore, due to the size and complexity of large-scale heterogeneous visual data, tools must also be able to facilitate human investigation in a way that can effectively support cognition (without overburdening the user). Recent developments in tangible user interfaces (TUI) and embodied cognition in particular, have identified a link between the use of motor skills and the enhanced cognition that it stimulates. Extending visual data mining paradigms to include such interfaces thus seems logical.******To this end, we aim to investigate such tools on several fronts: 1) through the detection of novelty in the data based on visual salience, which will be considered in terms of both 2D image, and 3D structural features; and 2) through the unsupervised structuring and organization of detected novelty across different spatial locations, and across different sources of heterogeneous visual data, in order to establish contextual descriptions indicating candidate locations and details in the data that can be targeted for investigation, comparison and analysis. Finally, 3) we will investigate the integration of these principles within tangible user interfaces and tangible query construction for visualization and exploration of rich visual data.******In the proposed program of research, we will first advance the state-of-the-art in 2D/3D visual salience as suitable descriptors for identifying informative details of interest. We will develop methodologies for systematic compartmentalization and characterization of visual novelty, and explore frameworks in which such novelty can be made readily accessible to the user in support of effective and targeted inquiry.