Multimedia terminal system for the blind using three-dimensional sound and tactile display

该项目旨在为盲人开发一个多媒体终端系统。一般来说，盲人使用像塑料片或浮雕图像这样的触觉图像来识别图像信息。此外，还有触觉显示系统，如大量使用针脚矩阵的盲文显示。触觉显示系统可以立即改变显示图像。我们的多媒体系统由触觉显示系统、一台计算机和一支触控笔组成，触控笔将触觉显示与光束连接起来。使用触控笔，盲人可以直接在触觉表面输入位置信息。这是该系统的主要特点。盲人用户可以在触觉图像上的任意位置点，这样系统就可以向用户反馈一种触觉或声音动作，如振动销钉、改变图像、播放声音等。我们做了一个绘图软件，乒乓球游戏，学习了一个汉字软件。通过绘图软件，盲人可以用触控笔在触觉显示器上画出线条，并可以擦除。在乒乓球比赛中，他找到了一个移动的球，用针和击球来表达，用触控笔控制球拍。通过汉字软件，他可以学习汉字中各个部分的意思和发音。当他指着汉字的某个部分时，这个部分就会闪烁。发音信息是通过按一个按钮给出的。一些视障学校的盲人对系统进行了评估。他们评论说，游戏是非常愉快的，学习系统是必要的。特别是老师们对汉字学习软件很感兴趣。另一方面，他们需要高分辨率的触觉显示。

The project aims to develop a multimedia terminal system for the blind. In general, blind persons use a tactile image like a plastic sheet or an embossed image to recognize image information. In addition, there are tactile display systems like a mass of Braille display using pin-matrixes. The tactile display system can change the display image immediately. Our multimedia system is composed of the tactile display system, a computer, and a stylus pen that connected the tactile display with beams. Using the stylus, the blind person can input position information to the tactile surface directly. That is the main feature of the system. The blind user can point arbitrary positions over the tactile image, so that the system can feedback to the user a kind of tactile or acoustic actions like vibrating pins, changing the image, playing a sound, and so on. We made a drawing software, ping-pong game, and learning a kanji software. With drawing software, the blind can make a line drawing with stylus on the tactile display and he can erase it. With ping-pong game, he finds a moving ball expressed by pins and strikes against using racket controlled by the stylus. With kanji software, he can learn the means and pronunciations of parts in a kanji character. When he pointed a part of a kanji, the part blinks. And the pronunciation information is given by pressing a button. The blind persons in some visually impaired school estimated the system. They comment that the game is very enjoyable and the learning system is necessary. Especially the teachers were interested in the kanji learning software. On the other hand, they required highly resolution for the tactile display.