Development of HMD having eye-gaze detection function and realization of advanced bilateral communication for ALS patients

这项研究的目的是使ALS(肌萎缩侧索硬化症)患者能够与周围的人交流，并通过输入字符和选择菜单选项来操作远程人性化的移动机器人，利用患者的剩余功能，一种视觉光线。它附着在一只眼睛上，其特点是，即使眼睛和头盔显示器发生偏差，它也能在头盔显示器屏幕上检测到准确的眼睛凝视点。来自PC的图像显示在屏幕上。同时，为眼睛照射LED的近红外光，并对眼睛图像进行抓取和处理，然后检测由LED光源产生的瞳孔中心和角膜反射。眼睛凝视点是由这些中心的相对位置计算出来的。在2005财年，试用产品中的眼睛凝视检测很困难，因为光源的鬼影图像落在了眼睛图像上。因此，在2000财年…此外，还对鬼影产生的原因进行了探讨。通过改变光源布置，解决了这一问题。然而，其总规模比预期的要大。此外，由于它们的离散性，眼睛凝视点的信号被平滑了。利用研制的头盔显示器作为移动机器人远程操作系统的I/O接口进行了远程控制实验。该机器人具有全方位移动机构，摄像头用于远程拍摄，距离传感器用于获取距离信息。HMD显示的操作屏幕分辨率较低，为640×480。然而，我们开发的操作界面包括远程环境地图、实时图像、操作机器人的按钮、操作摄像系统的按钮和摄像头方向的显示这五个最小功能。当将光标放置在机器人操作按钮上两秒钟时，考虑到安全性，进行按钮指示的操作。此外，考虑到检测凝视方向的特点，在操作屏幕的四个角上设置了停止按钮。通过运行实验，证实了移动机器人和摄像系统均可进行操作。较少

The purpose of this study is make it possible for ALS (amyotrophic lateral sclerosis) patients to communicate with surrounding people and to operate remote human-friendly mobile robots by inputting characters and selecting menu options using residual function of the patients, a light of sight.The HMD (head mounted display) has eye-gaze detection function. It is attached to one eye and characterized that it can detect the exact eye gaze point on the HMD screen in spite of deviation of the eye and the HMD. Image from a PC is presented on the screen. At the same time, irradiation near-infrared light of an LED for the eye, and grabbing and processing the eye image and then detecting the centers of the pupil and the corneal reflection produced by the LED light source. The eye gaze point is calculated by the relative position of these centers. In fiscal 2005, eye gaze detection was difficult in the trial product because ghost images of the light source fell on the eye image. So in fiscal 200 … More 6, the reason of the ghost images was investigated. By changing light source arrangement, the problem was solved. However, its total size was bigger than expected. Moreover, due to their dispersion the signal of eye gaze points was smoothed. However, the characteristics which can compensate eye gaze points against deviation of the eye and the HMD is demonstrated.The remote control experiment was carried out in which the developed HMD was used as an I/O interface in a remote operation system of mobile robot. The robot has omni-directional mobile mechanism with the camera to take pictures in remote site and the range sensor to acquire distance information. The operation screen shown by HMD is low resolution, 640 by 480. However, we developed the operation interface including five functions as a minimum function to operate the robot remotely - a remote environmental map, the live images, the button to operate robot, the button to operate camera system, and the display of camera direction. When cursor was placed for two seconds on the robot operation button the operation which the button indicates was made to be performed considering safety. Moreover taking care of the characteristic of detecting the direction of gaze, the stop buttons are arranged in the four corners of operation screen. It has been confirmed that both the mobile robot and the camera system could be operated from operation experiment. Less