Development of self-learning system for improving language ability of the hearing-impaired

作为提高听障人士语言能力的自主学习系统研究的一部分，开发了一个电子手语词典系统。本词典使用的手语词汇是日语手语中日常使用频率较高的950个手语词汇，旨在分析手势的细节，以求精炼描述，并用于初学者手语的自学。用手形的新描述性符号以及手的位置和运动的描述以及相应日语的语音和正字法描述来标记每个所选手势单词。这些文字的视频图片由一名女性本土签名者签名，记录在激光光盘上。手形描述符号系统建立在手指动作的解剖结构模型基础上，综合了手形的基本特征、食指、中指、无名指和小指之间的相互关系以及拇指的特征，并计划向数据库中引入句法信息，以使每个词与其他词相协调。除了连续单词的手势之间的连接之外，由诸如单词的持续时间、停顿和面部表情等韵律特征引起的短语和句子结构的组织将在下一阶段中使用。

As a part of study on self-learning system for improving language ability of the hearing-impaired, an electronic sign language dictionary system has been developed. The system is planned for use in analyzing the detail of sign gesture in order to refine the descriptions, as well as for use in self-learning of sign language by beginners.The sign language vocabulary used in this dictionary is a set of 950 sign words which are highly frequent in daily use in Japanese sign language. Each of the selected sign words is labeled with a new descriptive symbol for the hand shape, as well as a description of the position and movement of the hands, and phonetic and orthographic descriptions of corresponding Japanese. Video pictures of these words, signed by a female native signer, are recorded on a lazer disc. These recorded video pictures are linked to a computer controlled laser disc player, so that the corresponding video pictures in the sign language vocabulary can be retrieved instantaneously from any of the descriptive items in the database.The descriptive symbol system for the hand shape proposed here is based on the anatomical structure modelling of finger action, and consists of a combination of primary features of the hand shapes, the mutual relationships among the index, middle, ring and little fingers, and features of the thumb.It is also planned to introduce syntactic information to the database to coordinate each word with other words. In addition to the concatenation between the gestures for successive words, the organization of phase and sentence structure, caused by prosodic features such as the time duration of the words, pause, and facial expression ib to be utilized in the next stage.