Development of self-learning system for improving language ability of the hearing-impaired

点击翻译按钮获取中文摘要

As a part of study on self-learning system for improving language ability of the hearing-impaired, an electronic sign language dictionary system has been developed. The system is planned for use in analyzing the detail of sign gesture in order to refine the descriptions, as well as for use in self-learning of sign language by beginners.The sign language vocabulary used in this dictionary is a set of 950 sign words which are highly frequent in daily use in Japanese sign language. Each of the selected sign words is labeled with a new descriptive symbol for the hand shape, as well as a description of the position and movement of the hands, and phonetic and orthographic descriptions of corresponding Japanese. Video pictures of these words, signed by a female native signer, are recorded on a lazer disc. These recorded video pictures are linked to a computer controlled laser disc player, so that the corresponding video pictures in the sign language vocabulary can be retrieved instantaneously from any of the descriptive items in the database.The descriptive symbol system for the hand shape proposed here is based on the anatomical structure modelling of finger action, and consists of a combination of primary features of the hand shapes, the mutual relationships among the index, middle, ring and little fingers, and features of the thumb.It is also planned to introduce syntactic information to the database to coordinate each word with other words. In addition to the concatenation between the gestures for successive words, the organization of phase and sentence structure, caused by prosodic features such as the time duration of the words, pause, and facial expression ib to be utilized in the next stage.