Mechanisms of integrating haptic cues for VR-based interactive analytics of complex data

点击翻译按钮获取中文摘要

PROBLEM: Many scientific/engineering applications yield increasingly complex data in multiple dimensions and variates. Although application-specific algorithms can extract certain ill-defined patterns of the data, the intricate characteristics of these patterns require experts (users) with various competencies to analyze them interactively and collaboratively (i.e., interactive analytics) for decision-making. High-end computers usually display the patterns as 3-dimensional objects in intertwisted shapes (local attributes) and interlinkages (global relationships). Routinely, the users interact with the objects via keyboards/mice while viewing them on screens. This practice deters the users from conducting interactive analytics efficiently, due to difficulties in relying solely on the sense of vision for decoding the interconnection between the local attributes and global relationships. A solution might employ the sense of touch to explore the local attributes, while maintaining the view of the global relationships. The solution involves the human ability of using both senses to interact with objects for achieving cognitive responses. Rendering visual and haptic (touch by force and vibrotactile) cues to stimulate both senses, a computer platform based on advanced Virtual Reality (VR) technologies has a potential for catalyzing interactive analytics. Impediments are however two major deficiencies in the current knowledgebase: (a) the lack of robust schemes for associating the haptic cues with the local attributes of the patterns, and (b) the unavailability of adequate models for integrating the cues to foster neuro-ergonomic interaction. OBJECTIVE: The long-term objective of this program is to address the deficiencies by creating computational mechanisms for integrating the haptic cues, hence catalysing VR-based interactive analytics. The short-term objectives of the program are to establish schemes of force and vibrotactile cues (Theme A) and to build models of integrating the cues (Theme B). IMPACT: It is challenging to invent cue-integration mechanisms in complying with behavioural and neural responses for fostering neuro-ergonomic interaction. Collaborating with national and international researchers, the program meets the challenge by achieving two innovative elements: (i) a methodology of creating cue schemes from pattern attributes, and (ii) an optimization of ensuring the cue-integration. The achievement signifies a knowledge gain to permit the use of haptic cues for facilitating interactive analytics, and a technical advancement to enable a viable VR-based platform for benefiting Canadian economy by promoting experts' actions in applications including: planning public safety (e.g., flood mitigation), boosting industrial competitiveness (e.g., reactor design), innovating trustable human-machine interfaces (e.g., human-robot defense systems), and inventing practical brain-machine interaction (e.g., object manipulation via neural signals).