Study on gesture-based learning system with immersive virtual environments

本研究开发了控制虚拟空间的体位学习界面，旨在构建高效的体位学习系统。在沉浸式虚拟环境中使用什么样的界面是决定应用程序可用性的重要因素。因此，我们设计了无有线连接的人体位置检测界面系统，使用户可以直观地操纵沉浸式虚拟环境；开发了带有图像处理的控制界面，以流畅地在沉浸式虚拟环境中进行人体位置交互。使用图像处理使用户能够移动他们的手和腿，而不是有线连接。通过获取深度图像并对多个视点的视频图像进行处理，开发了跟踪用户头部和手臂位置的功能，使其适应于围绕用户的沉浸式虚拟环境系统。我们建立了用作接口的基础架构…在沉浸式投影显示中更多的是实时检测用户的身体位置，在虚拟环境中通过身体位置实现虚拟空间的控制界面以实现视点运动控制功能后，评估用户的表现。进行了用户测试，将界面与传统设备(即操纵杆)和手臂手势进行了比较。测试结果表明，该体位界面适合于在虚拟环境中快速移动，而不是正确超前于指定的路径；开发了基于体位交互的虚拟环境控制界面，研究了在虚拟环境中呈现星际等离子体和月球表面内容的可视化系统、作为虚拟环境显示设备的头盔显示器的可用性测试、构建虚拟共享空间的网络特性以及使用增强现实技术将虚拟对象覆盖到现实世界上的增强指令。较少

In this research, interface using body position, which controls virtual space, was developed, aiming to construct efficient learning system using the body position. What kinds of interface we use in immersive virtual environment is an important factor that dominates usability of applications. Thus, we designed the interface system for detecting body position without wired connection, so that a user can intuitively manipulate immersive virtual environments.The control interface with image processing was developed to smoothly make body position interaction in immersive virtual environments. Using the image processing enables users to move their hands and legs, instead of wired connection. The function of tracking position of the user's head and arms was developed by acquiring depth images with processing video images from plural viewpoints, adapting the function to the immersive virtual environment system that surrounds users. We established infrastructure that was used as an interface, … More detecting body position of the user in real time in immersive projection display.The user performance was evaluated after implementing the control interface of virtual space by body position to the function of viewpoint motion control in virtual environments. The user test was performed, comparing the interface with conventional device (i.e., joystick) and hand-arm gestures. The test result showed that the body position interface was suitable for moving around the virtual environments fast rather than preceding the designated path correctly.When the control interface of virtual environments by body position interaction was developed, the following researches are done for the basic knowledge or their applications ; the visualization systems that presented the contents of interplanetary plasma and lunar surface in the virtual environment, usability test of HMD that makes us hands-free presentation as a display device of virtual environments, the network characteristics for constructing a virtual shared space, and augmented instructions that uses augmented reality technology overlaying virtual objects onto the real world. Less