A Study on automatic construction of virtual space from natural scenes and non-contact access

我们提出并构建了“3D编辑器”，通过它我们可以在计算机中访问由自然场景组成的虚拟空间。我们开发了以下三种算法作为本系统的关键技术。(1)从自然场景构建虚拟空间我们开发了一个系统，该系统通过在直线上放置（或移动）的多个摄像机输入多视点图像集，获取三维自然场景的深度信息。该系统通过将三维结构划分为不同视差的多层，并从近到远校正遮挡导致的错误视差，从而准确获取深度信息。此外，我们还开发了一种通过插值获得的深度信息来组成虚拟三维空间的算法。(2)基于手势的人机界面构建构建了一个紧凑的基于手势的人机界面。手势是用来访问虚拟的三维空间。显示器顶部设置了两个摄像头，用户通过在显示器前移动手来输入信息。在工作站中分析手的立体运动场景，检测手在三维空间中的运动。在该系统中，检测手的运动和形式独立于用户的衣服和背景。(3)通过手势输入命令我们开发了一种算法，可以从检测到的手部三维运动中识别命令。在提议系统中，通过将连续操作分解为单个操作单元，并将单元与模型匹配来识别手势。该算法可以解释各种三维命令。

We proposed and constructed the "3D editor" with which we can access the virtual space composed from a natural scene in the computer.We developed the following three algorithm as key technologies of this system.(1) Construction of virtual space from natural scenesWe developed a system with which the depth information of three dimensional natural scenes is acquired from a multi-viewpoint image set which is input through several cameras placed (or one camera moving) on a straight line.In the proposal system the depth information is obtained accurately by dividing the three dimensional structure into multi layrs with different disparity, and correcting a wrong disparity caused by occlusion from near to far.In addition, we developed an algorithm which composes the virtual three dimensional space by interpolating the obtained depth information.(2) Construction of gesture-based man-machine interfaceWe cunstructed a compact gesture-based man-machine interface. Hand gesture is used to access the virtual three dimensional space.Two cameras are set on the top of the monitor and users input information by moving their hand in front of the monitor.The stereo moving scene of the hand is analyzed in the workstation and the hand motion in the three dimensional space is detected.In the proposed system, the motion and form of the hand are detetected indeopendently of user's clothes and background.(3) Command input by gestureWe developed an algorithm to recognize commands from the detected three dimensional motion of the hand. In the proposal system, the gesture is recognized by decomposing a continuous operation into individual operation units, and matching the unit with the model.Various three dimensional commands can be interpreted by this algorithm.