A hands-free interactive system for manipulating object-oriented information within networked virtual environments

虚拟现实（VR）旨在通过视觉和触觉等多感官形式进行实时仿真和交互，在石油和天然气勘探等领域有着广泛的应用。然而，由于与直观用户交互相关的瓶颈问题，VR应用程序的适应一直很缓慢。当VR应用程序在计算机网络上运行以进行多用户交互时，这个问题变得更加严重。我们建议创建一个免提交互系统，用于在网络虚拟环境中操纵面向对象的信息。我们将开发独立的桌面虚拟现实，并将它们互连以形成网络虚拟环境。每个桌面VR将生成油藏模拟的3D立体显示-计算机建模集团有限公司（CMG）的软件产品。桌面VR的基础将是一个免提交互系统。该交互系统将利用基于相机的手势识别（提取用户与3D显示器的对象交互的意图）和触觉（通过向用户的手提供力反馈来模拟触觉）。整合手势识别和触觉的框架将采用人类用于执行视觉感知和视觉引导动作的神经机制。交互系统将允许多个用户在网络虚拟环境中协作他们的个人交互（通过每个桌面VR）。该项目的新奇之处在于将手势识别和触觉技术集成到用户交互中;使用新的3D测距相机进行手势识别;以及应用神经机制来解决工程问题。我们以前的工作证实，这是可以实现的，利用现有的技术和知识，创造免提交互系统。作为储层模拟的新工具，免提交互系统将推动加拿大有效的石油相关服务和经济发展。它的好处将是允许CMG领导一个利基市场的油藏模拟。其影响将超越石油相关服务，成为连接和增强其他应用的创新VR技术的桥梁。

Aiming at real-time simulation and interaction via multi-sensory modalities such as vision and touch, virtual reality (VR) finds many applications including oil and gas exploration. However, adaptation of VR applications has been slow, due to a bottle-neck issue related to intuitive user interaction. This issue becomes more problematic when VR applications run across a computer network for multi-user interaction. We propose to create a hands-free interactive system for manipulating object-oriented information within networked virtual environments. We will develop self-contained desktop VRs and interconnect them to form a networked virtual environment. Each desktop VR will generate 3D stereoscopic displays of the Reservoir Simulation - a software product from the Computer Modelling Group Ltd (CMG). Underpinning the desktop VR will be a hands-free interactive system. This interactive system will utilize camera-based gesture recognition (extracting the user intention of interacting with objects of the 3D displays) and haptics (simulating the sense of touch by providing force feedback to the user's hand). The framework of integrating both gesture recognition and haptics will employ the neural mechanisms that humans use to perform visual perception and visually guided actions. The interactive system will allow multiple users to collaborate their individual interaction (via each desktop VR) across the networked virtual environment. The novelty of the project lies in the integration of gesture recognition and haptics for user interaction; the use of new 3D range cameras for gesture recognition; and the application of the neural mechanisms to address engineering issues. Our previous work confirms that it is achievable to create the hands-free interactive system using current technologies and knowledge. As a novel tool for reservoir simulation, the hands-free interactive system will propel effective oil-related services and economic development for Canada. Its benefit will be allowing the CMG to lead a niche market of reservoir simulation. Its impact will go beyond the oil-related services, serving as a bridge to connect and enhance innovative VR-based technologies for other applications.