A Study for Manipulation and Editing System with Multi Modal Interface

点击翻译按钮获取中文摘要

In this research, we discussed the method of cooperative virtual reality system which harmonize the virtual objects and real world. The proposed system does not model all the real scene into the virtual world, but take it in to the system as the images. Moreover, virtual objects are generated and edited with the use of multimodal interface system. Our goal is to establish the essential technology for the proposed system. The main results are as follows :Manipulation in the virtual nature environmentAs a first step towards the system, in which the user is able to operate plant models inside the virtual environment freely with natural interface, we explore the method about dynamic object data reconstruction. So, we can arrange the operations of plant in order and investigate the method that is able to divide plant models according to their parts.Navigation in an enormous virtual environmentWe propose a manipulation technique to intuitively control the "bird's eye" overview display of an entire large-scale virtual environment in a display system that present a user with both overviews (global views) and a life-size virtual environment (local view) simultaneously. It enables efficient navigation even in enormous and complicated environments using both global and local views. The motion of the bird's eye viewpoint interlocks with the relative motion of the user's viewpoint and his/her hand, therefore, the user can control the "bird's eye" viewpoint by intuitive manipulation.Panoramic real images for real-time walk-throughWe proposed a method to create a realistic distant scenery in a virtual environment using panoramic real images taken by an omnidirectional stereo imaging sensor. The system can represent motion parallax following viewpoints according to user's movement.