Study on gesture-based learning system with immersive virtual environments

在本研究中，开发了控制虚拟空间的身体姿势界面，旨在构建有效的身体姿势学习系统。在沉浸式虚拟环境中使用什么样的界面是决定应用程序可用性的一个重要因素。为此，我们设计了无需有线连接的人体位置检测接口系统，使用户能够直观地操作沉浸式虚拟环境，并开发了具有图像处理功能的控制接口，使人体位置在沉浸式虚拟环境中的交互更加流畅。使用图像处理使用户能够移动他们的手和腿，而不是有线连接。通过从多个视点处理视频图像来获取深度图像，开发了跟踪用户的头部和手臂的位置的功能，使该功能适合于围绕用户的沉浸式虚拟环境系统。我们建立了用作接口的基础设施， 关于我们 在沉浸式投影显示中，实时真实的检测用户的身体位置，并对虚拟空间的身体位置控制接口实现虚拟环境中视点运动控制功能后的用户表现进行评价。进行了用户测试，将接口与传统设备（即，操纵杆）和手臂手势。测试结果表明，该体位交互界面适合于在虚拟环境中快速移动，而不是按照指定的路径正确行进。在虚拟环境中显示行星际等离子体和月球表面内容的可视化系统，HMD的可用性测试，使我们免提演示作为虚拟环境的显示设备，用于构建虚拟共享空间的网络特性，以及使用增强现实技术将虚拟对象叠加到真实的世界上的增强指令。少

In this research, interface using body position, which controls virtual space, was developed, aiming to construct efficient learning system using the body position. What kinds of interface we use in immersive virtual environment is an important factor that dominates usability of applications. Thus, we designed the interface system for detecting body position without wired connection, so that a user can intuitively manipulate immersive virtual environments.The control interface with image processing was developed to smoothly make body position interaction in immersive virtual environments. Using the image processing enables users to move their hands and legs, instead of wired connection. The function of tracking position of the user's head and arms was developed by acquiring depth images with processing video images from plural viewpoints, adapting the function to the immersive virtual environment system that surrounds users. We established infrastructure that was used as an interface, … More detecting body position of the user in real time in immersive projection display.The user performance was evaluated after implementing the control interface of virtual space by body position to the function of viewpoint motion control in virtual environments. The user test was performed, comparing the interface with conventional device (i.e., joystick) and hand-arm gestures. The test result showed that the body position interface was suitable for moving around the virtual environments fast rather than preceding the designated path correctly.When the control interface of virtual environments by body position interaction was developed, the following researches are done for the basic knowledge or their applications ; the visualization systems that presented the contents of interplanetary plasma and lunar surface in the virtual environment, usability test of HMD that makes us hands-free presentation as a display device of virtual environments, the network characteristics for constructing a virtual shared space, and augmented instructions that uses augmented reality technology overlaying virtual objects onto the real world. Less