Study on gesture-based learning system with immersive virtual environments

本研究开发了控制虚拟空间的基于身体位置的界面，旨在构建高效的基于身体位置的学习系统。在沉浸式虚拟环境中使用什么样的界面是决定应用程序可用性的一个重要因素。因此，我们设计了无需有线连接即可检测人体位置的界面系统，使用户可以直观地操作沉浸式虚拟环境。为了在沉浸式虚拟环境中实现流畅的身体位置交互，开发了图像处理控制界面。使用图像处理，用户可以移动他们的手和腿，而不是有线连接。通过对多视点视频图像进行处理，获取深度图像，开发了用户头部和手臂位置跟踪功能，使该功能适应用户周围的沉浸式虚拟环境系统。我们建立了作为界面的基础设施，在沉浸式投影显示中实时检测用户的身体位置。在虚拟环境中，通过身体位置实现虚拟空间的控制界面，对视点运动控制功能进行用户性能评价。进行用户测试，将界面与传统设备（即操纵杆）和手臂手势进行比较。测试结果表明，人体位置界面适合在虚拟环境中快速移动，而不是正确地走在指定的路径上。在开发基于身体位置交互的虚拟环境控制界面时，对其基础知识及其应用进行了以下研究：在虚拟环境中呈现行星际等离子体和月球表面内容的可视化系统，作为虚拟环境显示设备实现免提展示的HMD可用性测试，构建虚拟共享空间的网络特性，以及使用增强现实技术将虚拟物体叠加到现实世界中的增强指令。少

In this research, interface using body position, which controls virtual space, was developed, aiming to construct efficient learning system using the body position. What kinds of interface we use in immersive virtual environment is an important factor that dominates usability of applications. Thus, we designed the interface system for detecting body position without wired connection, so that a user can intuitively manipulate immersive virtual environments.The control interface with image processing was developed to smoothly make body position interaction in immersive virtual environments. Using the image processing enables users to move their hands and legs, instead of wired connection. The function of tracking position of the user's head and arms was developed by acquiring depth images with processing video images from plural viewpoints, adapting the function to the immersive virtual environment system that surrounds users. We established infrastructure that was used as an interface, … More detecting body position of the user in real time in immersive projection display.The user performance was evaluated after implementing the control interface of virtual space by body position to the function of viewpoint motion control in virtual environments. The user test was performed, comparing the interface with conventional device (i.e., joystick) and hand-arm gestures. The test result showed that the body position interface was suitable for moving around the virtual environments fast rather than preceding the designated path correctly.When the control interface of virtual environments by body position interaction was developed, the following researches are done for the basic knowledge or their applications ; the visualization systems that presented the contents of interplanetary plasma and lunar surface in the virtual environment, usability test of HMD that makes us hands-free presentation as a display device of virtual environments, the network characteristics for constructing a virtual shared space, and augmented instructions that uses augmented reality technology overlaying virtual objects onto the real world. Less