Mechanisms of integrating haptic cues for VR-based interactive analytics of complex data

问题：许多科学/工程应用产生越来越复杂的多维和变量数据。虽然特定于应用程序的算法可以提取数据的某些定义不清的模式，但这些模式的复杂特征需要具有各种能力的专家（用户）以交互和协作（即交互式分析）的方式进行分析，以做出决策。高端计算机通常将模式显示为具有交错形状（局部属性）和相互联系（全局关系）的三维对象。通常，用户通过键盘/鼠标与对象进行交互，同时在屏幕上查看它们。这种做法阻碍了用户有效地进行交互式分析，因为仅仅依靠视觉来解码本地属性和全局关系之间的互连是很困难的。解决方案可以使用触觉来探索局部属性，同时保持全局关系的视图。解决方案涉及到人类使用两种感官与物体互动以实现认知反应的能力。一个基于先进虚拟现实（VR）技术的计算机平台，通过渲染视觉和触觉（通过力和振动触觉）线索来刺激两种感官，具有催化交互式分析的潜力。然而，目前的知识库存在两个主要缺陷：(a)缺乏将触觉线索与模式的局部属性相关联的稳健方案，以及(b)没有足够的模型来整合线索以促进神经人机交互。目的：该项目的长期目标是通过创建集成触觉线索的计算机制来解决缺陷，从而促进基于vr的交互式分析。该项目的短期目标是建立力和振动触觉线索（主题A）的方案，并建立整合这些线索的模型（主题B）。影响：这是具有挑战性的发明线索整合机制，以符合行为和神经反应，促进神经人机工程学的相互作用。与国内外研究人员合作，该项目通过实现两个创新元素来应对挑战：(i)从模式属性创建线索方案的方法，以及（ii）确保线索整合的优化。这一成就标志着知识的获得，允许使用触觉线索来促进交互式分析，以及技术的进步，使一个可行的基于vr的平台通过促进专家在应用中的行动来造福加拿大经济，包括：规划公共安全（例如，防洪减灾），提高工业竞争力（例如，反应堆设计），创新可信赖的人机界面（例如，人-机器人防御系统），并发明实用的脑-机交互（例如，通过神经信号操纵物体）。

PROBLEM: Many scientific/engineering applications yield increasingly complex data in multiple dimensions and variates. Although application-specific algorithms can extract certain ill-defined patterns of the data, the intricate characteristics of these patterns require experts (users) with various competencies to analyze them interactively and collaboratively (i.e., interactive analytics) for decision-making. High-end computers usually display the patterns as 3-dimensional objects in intertwisted shapes (local attributes) and interlinkages (global relationships). Routinely, the users interact with the objects via keyboards/mice while viewing them on screens. This practice deters the users from conducting interactive analytics efficiently, due to difficulties in relying solely on the sense of vision for decoding the interconnection between the local attributes and global relationships. A solution might employ the sense of touch to explore the local attributes, while maintaining the view of the global relationships. The solution involves the human ability of using both senses to interact with objects for achieving cognitive responses. Rendering visual and haptic (touch by force and vibrotactile) cues to stimulate both senses, a computer platform based on advanced Virtual Reality (VR) technologies has a potential for catalyzing interactive analytics. Impediments are however two major deficiencies in the current knowledgebase: (a) the lack of robust schemes for associating the haptic cues with the local attributes of the patterns, and (b) the unavailability of adequate models for integrating the cues to foster neuro-ergonomic interaction. OBJECTIVE: The long-term objective of this program is to address the deficiencies by creating computational mechanisms for integrating the haptic cues, hence catalysing VR-based interactive analytics. The short-term objectives of the program are to establish schemes of force and vibrotactile cues (Theme A) and to build models of integrating the cues (Theme B). IMPACT: It is challenging to invent cue-integration mechanisms in complying with behavioural and neural responses for fostering neuro-ergonomic interaction. Collaborating with national and international researchers, the program meets the challenge by achieving two innovative elements: (i) a methodology of creating cue schemes from pattern attributes, and (ii) an optimization of ensuring the cue-integration. The achievement signifies a knowledge gain to permit the use of haptic cues for facilitating interactive analytics, and a technical advancement to enable a viable VR-based platform for benefiting Canadian economy by promoting experts' actions in applications including: planning public safety (e.g., flood mitigation), boosting industrial competitiveness (e.g., reactor design), innovating trustable human-machine interfaces (e.g., human-robot defense systems), and inventing practical brain-machine interaction (e.g., object manipulation via neural signals).