A Study for Manipulation and Editing System with Multi Modal Interface

在本研究中，我们探讨了协调虚拟物体与现实世界的协同虚拟现实系统的方法。所提出的系统不是将所有真实场景都建模到虚拟世界中，而是将其作为图像引入系统中。并利用多通道界面系统生成和编辑虚拟对象。我们的目标是为拟议的系统建立关键技术。主要研究结果如下：虚拟自然环境中的操作作为系统的第一步，用户可以通过自然界面在虚拟环境中自由地操作植物模型，探索了动态对象数据重建的方法。因此，我们可以对植物的操作进行有序的安排，并研究能够按部分划分植物模型的方法。在巨大的虚拟环境中导航我们提出了一种操纵技术，在显示系统中直观地控制整个大规模虚拟环境的鸟瞰显示，同时向用户呈现全景(全局视图)和真人大小的虚拟环境(局部视图)。即使在巨大和复杂的环境中，它也可以使用全局和局部视图实现高效导航。鸟眼视点的运动与用户视点和手的相对运动相关联，因此用户可以通过直观的操作来控制鸟眼视点。实时漫游的全景实时图像提出了一种利用全方位立体成像传感器拍摄的全景真实图像在虚拟环境中创建逼真的远处风景的方法。该系统可以根据用户的运动来表示视点跟随视点的运动视差。

In this research, we discussed the method of cooperative virtual reality system which harmonize the virtual objects and real world. The proposed system does not model all the real scene into the virtual world, but take it in to the system as the images. Moreover, virtual objects are generated and edited with the use of multimodal interface system. Our goal is to establish the essential technology for the proposed system. The main results are as follows :Manipulation in the virtual nature environmentAs a first step towards the system, in which the user is able to operate plant models inside the virtual environment freely with natural interface, we explore the method about dynamic object data reconstruction. So, we can arrange the operations of plant in order and investigate the method that is able to divide plant models according to their parts.Navigation in an enormous virtual environmentWe propose a manipulation technique to intuitively control the "bird's eye" overview display of an entire large-scale virtual environment in a display system that present a user with both overviews (global views) and a life-size virtual environment (local view) simultaneously. It enables efficient navigation even in enormous and complicated environments using both global and local views. The motion of the bird's eye viewpoint interlocks with the relative motion of the user's viewpoint and his/her hand, therefore, the user can control the "bird's eye" viewpoint by intuitive manipulation.Panoramic real images for real-time walk-throughWe proposed a method to create a realistic distant scenery in a virtual environment using panoramic real images taken by an omnidirectional stereo imaging sensor. The system can represent motion parallax following viewpoints according to user's movement.