Development of HMD having eye-gaze detection function and realization of advanced bilateral communication for ALS patients

点击翻译按钮获取中文摘要

The purpose of this study is make it possible for ALS (amyotrophic lateral sclerosis) patients to communicate with surrounding people and to operate remote human-friendly mobile robots by inputting characters and selecting menu options using residual function of the patients, a light of sight.The HMD (head mounted display) has eye-gaze detection function. It is attached to one eye and characterized that it can detect the exact eye gaze point on the HMD screen in spite of deviation of the eye and the HMD. Image from a PC is presented on the screen. At the same time, irradiation near-infrared light of an LED for the eye, and grabbing and processing the eye image and then detecting the centers of the pupil and the corneal reflection produced by the LED light source. The eye gaze point is calculated by the relative position of these centers. In fiscal 2005, eye gaze detection was difficult in the trial product because ghost images of the light source fell on the eye image. So in fiscal 200 … More 6, the reason of the ghost images was investigated. By changing light source arrangement, the problem was solved. However, its total size was bigger than expected. Moreover, due to their dispersion the signal of eye gaze points was smoothed. However, the characteristics which can compensate eye gaze points against deviation of the eye and the HMD is demonstrated.The remote control experiment was carried out in which the developed HMD was used as an I/O interface in a remote operation system of mobile robot. The robot has omni-directional mobile mechanism with the camera to take pictures in remote site and the range sensor to acquire distance information. The operation screen shown by HMD is low resolution, 640 by 480. However, we developed the operation interface including five functions as a minimum function to operate the robot remotely - a remote environmental map, the live images, the button to operate robot, the button to operate camera system, and the display of camera direction. When cursor was placed for two seconds on the robot operation button the operation which the button indicates was made to be performed considering safety. Moreover taking care of the characteristic of detecting the direction of gaze, the stop buttons are arranged in the four corners of operation screen. It has been confirmed that both the mobile robot and the camera system could be operated from operation experiment. Less