Development of HMD having eye-gaze detection function and realization of advanced bilateral communication for ALS patients

本研究的目的是使ALS（肌萎缩性侧索硬化症）患者能够与周围的人交流，并利用患者的残余功能，即视力，通过输入字符和选择菜单选项来操作远程人性化移动机器人。HMD（头戴式显示器）具有眼睛注视检测功能。它附着在一只眼睛上，其特点是即使眼睛与HMD偏离，也能检测到眼睛在HMD屏幕上的准确注视点。来自PC的图像显示在屏幕上。同时，对眼睛照射LED的近红外光，对眼睛图像进行抓取和处理，然后检测由LED光源产生的瞳孔中心和角膜反射。眼睛注视点是通过这些中心的相对位置来计算的。在2005财政年度，由于光源的幽灵图像落在眼睛图像上，因此在试验产品中眼睛注视检测很困难。因此，在2000财年，对鬼影的原因进行了调查。通过改变光源排列，解决了这一问题。然而，它的总规模大于预期。此外，由于注视点的弥散性，对注视点信号进行了平滑处理。然而，证明了眼睛注视点补偿眼睛和头戴式显示器偏差的特性。进行了远程控制实验，将所开发的HMD作为移动机器人远程操作系统的I/O接口。该机器人具有全向移动机构，具有用于远距离拍摄的相机和用于获取距离信息的距离传感器。HMD显示的操作画面分辨率较低，为640 * 480。然而，我们开发了操作界面，其中包括五个功能作为远程操作机器人的最小功能-远程环境地图，实时图像，操作机器人的按钮，操作摄像机系统的按钮，以及摄像机方向显示。当光标在机器人操作按钮上停留两秒钟后，该按钮所指示的操作即为安全操作。并且考虑到检测凝视方向的特点，在操作画面的四个角落设置了停止按钮。通过操作实验，证实了移动机器人和摄像系统都是可以操作的。少

The purpose of this study is make it possible for ALS (amyotrophic lateral sclerosis) patients to communicate with surrounding people and to operate remote human-friendly mobile robots by inputting characters and selecting menu options using residual function of the patients, a light of sight.The HMD (head mounted display) has eye-gaze detection function. It is attached to one eye and characterized that it can detect the exact eye gaze point on the HMD screen in spite of deviation of the eye and the HMD. Image from a PC is presented on the screen. At the same time, irradiation near-infrared light of an LED for the eye, and grabbing and processing the eye image and then detecting the centers of the pupil and the corneal reflection produced by the LED light source. The eye gaze point is calculated by the relative position of these centers. In fiscal 2005, eye gaze detection was difficult in the trial product because ghost images of the light source fell on the eye image. So in fiscal 200 … More 6, the reason of the ghost images was investigated. By changing light source arrangement, the problem was solved. However, its total size was bigger than expected. Moreover, due to their dispersion the signal of eye gaze points was smoothed. However, the characteristics which can compensate eye gaze points against deviation of the eye and the HMD is demonstrated.The remote control experiment was carried out in which the developed HMD was used as an I/O interface in a remote operation system of mobile robot. The robot has omni-directional mobile mechanism with the camera to take pictures in remote site and the range sensor to acquire distance information. The operation screen shown by HMD is low resolution, 640 by 480. However, we developed the operation interface including five functions as a minimum function to operate the robot remotely - a remote environmental map, the live images, the button to operate robot, the button to operate camera system, and the display of camera direction. When cursor was placed for two seconds on the robot operation button the operation which the button indicates was made to be performed considering safety. Moreover taking care of the characteristic of detecting the direction of gaze, the stop buttons are arranged in the four corners of operation screen. It has been confirmed that both the mobile robot and the camera system could be operated from operation experiment. Less