Context-Driven Salience for Querying and Visualizing Immersive 3D Content

点击翻译按钮获取中文摘要

Steady advances in the fields of surveying and remote sensing, 3D modelling acquisition and reconstruction, and digital-content delivery, offer new opportunities for researchers, students, and the general public to generate or obtain rich 2D and 3D visual data pertaining to various aspects of the world we live in. ******To date, much progress has been made in the sensing and acquisition of rich visual, spatially associated data (i.e. 3D scans, models and associated source imagery), their efficient representation, visualization and even physical reproduction (e.g. 3D printing). However, in areas such as cultural heritage or in establishing a digital record that can support general scientific enquiry, there is a pressing need for tools and methodologies that support the formation of queries to extract, analyse and dissect. ******Furthermore, due to the size and complexity of large-scale heterogeneous visual data, tools must also be able to facilitate human investigation in a way that can effectively support cognition (without overburdening the user). Recent developments in tangible user interfaces (TUI) and embodied cognition in particular, have identified a link between the use of motor skills and the enhanced cognition that it stimulates. Extending visual data mining paradigms to include such interfaces thus seems logical.******To this end, we aim to investigate such tools on several fronts: 1) through the detection of novelty in the data based on visual salience, which will be considered in terms of both 2D image, and 3D structural features; and 2) through the unsupervised structuring and organization of detected novelty across different spatial locations, and across different sources of heterogeneous visual data, in order to establish contextual descriptions indicating candidate locations and details in the data that can be targeted for investigation, comparison and analysis. Finally, 3) we will investigate the integration of these principles within tangible user interfaces and tangible query construction for visualization and exploration of rich visual data.******In the proposed program of research, we will first advance the state-of-the-art in 2D/3D visual salience as suitable descriptors for identifying informative details of interest. We will develop methodologies for systematic compartmentalization and characterization of visual novelty, and explore frameworks in which such novelty can be made readily accessible to the user in support of effective and targeted inquiry.