Relic restoration using stereoscopic vision system with eye following function

在VR系统中，为了使立体显示产生的虚拟物体看起来像真实物体，观看者的眼睛位置和视线是必要的。在使用平面显示器生成基于双目视差的立体图像的情况下，需要相对于显示器的眼睛位置和视觉线信息。我们对我们提出的独特的摄像机标定方法“双圆算法”进行了扩展，实现了一种仅使用虹膜轮廓而不依赖于头部姿态的视觉方向估计方法。本研究已在多个国际会议（ECCV'04 FG'04和ICPR'04）和日本的一些会议（PRMU'0402， MIRU'04）上发表。为了实现双眼视差和运动视差的效果，必须根据观看者的眼睛相对于立体显示器的位置生成立体图像。为了获得这些参数，必须在立体显示器和相机之间进行校准，以估计眼睛的位置。但是，由于相机是用来获取观看者眼睛的信息；在一般情况下，显示器是不可见的。为了解决这个问题，我们让用户凝视显示器上显示的标记并用手指指向它。我们使用立体摄像机来估计眼睛和指尖的3D位置。立体相机和显示器之间的校准是利用制造者、指尖和眼睛在同一条线上的事实进行的。在此基础上，我们搭建了一个具有眼球跟随功能的立体显示系统，并将在MIRU2006上发表。

In a VR system, the eye position and the visual line of the viewer are necessary to make virtual objects generated by a stereo display look like a real objects. In the case of using a planar display to generate stereo images based on binocular parallax, the eye position and the visual line information relative to the display are required. We extended the "two-circle algorithm", which is an unique method for camera calibration proposed by us, to realize a visual direction estimation method that only uses the iris contours and is independent to the head pose. This research was reported at several international conferences (ECCV'04 FG'04 and ICPR'04) and some conferences held in Japan (PRMU'0402, MIRU'04).In order to realize the effects of binocular parallax and motion parallax, the stereo images have to be generated according to the eye positions of the viewer relative to the stereo display. To obtain those parameters the calibration between the stereo display and the camera for estimating the eye positions have to be carried out. However, since the camera is used for obtain the information about the viewer's eyes ; the display is not viewable in general cases. To solve this problem, we let the user gaze the marker shown on the display and point it with his/her finger. We use a stereo camera to estimate the 3D positions of the eye and the finger tip. The calibration between the stereo camera and the display is carried out by making use of the fact that the maker, the finger tip and the eye are on the same line. We built a stereo display system with eye following function based on this idea which will be reported on MIRU2006.