Context-Driven Salience for Querying and Visualizing Immersive 3D Content

测量和遥感、3D建模获取和重建以及数字内容交付领域的稳步发展为研究人员、学生和公众提供了新的机会，可以生成或获取与我们生活的世界各个方面有关的丰富的2D和3D视觉数据。 迄今为止，在感测和获取丰富的视觉、空间相关数据（即3D扫描、模型和相关源图像）、其有效表示、可视化甚至物理复制（例如3D打印）方面已经取得了很大进展。然而，在文化遗产等领域，或在建立可支持一般科学调查的数字记录方面，迫切需要有助于形成查询以进行提取、分析和剖析的工具和方法。 此外，由于大规模异构视觉数据的大小和复杂性，工具还必须能够以有效支持认知的方式促进人类调查（而不会使用户负担过重）。有形用户界面（TUI）和具体的认知，特别是最近的发展，已经确定了运动技能的使用和增强的认知，它刺激之间的联系。因此，扩展可视化数据挖掘范式以包括这样的接口似乎是合乎逻辑的。 为此，我们的目标是在几个方面研究此类工具：1）通过基于视觉显着性的数据新奇性检测，这将在2D图像和3D结构特征方面进行考虑;以及2）通过跨不同的空间位置，以及跨不同的异质视觉数据源，以便建立指示数据中的候选位置和细节的上下文描述，这些位置和细节可以作为调查、比较和分析的目标。最后，3）我们将研究这些原则在有形用户界面和有形查询构造中的集成，以实现丰富的可视化数据的可视化和探索。 在拟议的研究计划中，我们将首先推进最先进的2D/3D视觉显着性作为合适的描述符，用于识别感兴趣的信息细节。我们将开发视觉新奇的系统划分和表征方法，并探索框架，使这种新奇可以随时访问用户，以支持有效和有针对性的查询。

Steady advances in the fields of surveying and remote sensing, 3D modelling acquisition and reconstruction, and digital-content delivery, offer new opportunities for researchers, students, and the general public to generate or obtain rich 2D and 3D visual data pertaining to various aspects of the world we live in. To date, much progress has been made in the sensing and acquisition of rich visual, spatially associated data (i.e. 3D scans, models and associated source imagery), their efficient representation, visualization and even physical reproduction (e.g. 3D printing). However, in areas such as cultural heritage or in establishing a digital record that can support general scientific enquiry, there is a pressing need for tools and methodologies that support the formation of queries to extract, analyse and dissect. Furthermore, due to the size and complexity of large-scale heterogeneous visual data, tools must also be able to facilitate human investigation in a way that can effectively support cognition (without overburdening the user). Recent developments in tangible user interfaces (TUI) and embodied cognition in particular, have identified a link between the use of motor skills and the enhanced cognition that it stimulates. Extending visual data mining paradigms to include such interfaces thus seems logical. To this end, we aim to investigate such tools on several fronts: 1) through the detection of novelty in the data based on visual salience, which will be considered in terms of both 2D image, and 3D structural features; and 2) through the unsupervised structuring and organization of detected novelty across different spatial locations, and across different sources of heterogeneous visual data, in order to establish contextual descriptions indicating candidate locations and details in the data that can be targeted for investigation, comparison and analysis. Finally, 3) we will investigate the integration of these principles within tangible user interfaces and tangible query construction for visualization and exploration of rich visual data. In the proposed program of research, we will first advance the state-of-the-art in 2D/3D visual salience as suitable descriptors for identifying informative details of interest. We will develop methodologies for systematic compartmentalization and characterization of visual novelty, and explore frameworks in which such novelty can be made readily accessible to the user in support of effective and targeted inquiry.