A Study on Usages of Non-verbal Information for a Design of Information Assistive Tools

我们研究了如何利用非语言信息，以改进我们为视觉和听觉障碍者设计的两种信息获取工具。其中一个工具是一个新的屏幕阅读器系统，它将屏幕上的文本转换为合成语音，同时通过使用触觉刻度盘（TAJODA）在指尖表面显示丰富的文本信息，即非语言信息。这是一个为视障人士提供网络信息获取的辅助工具。另一个工具是一种新的助听装置（字幕系统），它可以自动将语音转换为文本，同时显示说话者的嘴唇运动和面部表情，即非语言信息，为获得性听障人士。关于“屏幕阅读器”的研究结果如下。(1)由于我们的研究结果表明盲人的最大听力速度是正常人的2.6倍，因此合成语音的速度应提高到正常说话速度的3倍。(2)传输富文本的最优数量为7条，即：字体大小（大于或小于标准）、彩色字母、粗体、括号（左或右）和段落。(3) 8名盲人用户的实验结果表明，网络句子的习得速度比普通屏幕阅读器快2.6倍。关于“字幕系统”的研究成果如下：(1)研究了说话人的面部图像和嘴唇运动（非语言信息）对句子理解的作用。(2)实验结果表明，嘴唇运动比面部表情更有效，并且从字幕开始的嘴唇运动显示延迟约2秒适合于理解的吸引。(3)将该字幕系统与非语言信息相结合，在国际会议上进行了约30次的试验，证明了该系统对理解有明显的促进作用。非语言信息；TAJODA中的触觉刺激和字幕系统中的唇部运动对内容的理解有明显的促进作用。此外，我们还可以确定非语言信息的最佳显示时间参数和显示方法，以改进为感官障碍者提供的信息辅助工具。少

We investigated how non-verbal information should be utilized in order to improve two information acquisition tools which we designed for the visually and auditory impaired.1.One of the tools is a new screen reader system which converts texts on the screen into synthesized speech while displaying rich text information, i.e. non-verbal information, onto a finger-tip surface by using a tactile jog-dial (TAJODA). This is a supporting tool of a web-information acquisition for visually impaired people.2.The other tool is a new hearing assistive device (captioning system) which automatically converts speech into texts while displaying speaker's lip-movement and facial expression, i.e. non-verbal information, for the acquired hearing impaired.Research results regarding "the screen reader" are as followings.(1)The speed of synthesized speech should be changed up to 3 times faster than normal speaking rate since our findings showed the maximum listening speeds for the blind is 2.6 times faster … More than the normal persons.(2)The optimal number of rich texts to transmit is 7;i.e. font sizes (larger or smaller than the standard), colored letter, bold, parentheses (left or right), and paragraph.(3)Experimental results by 8 blind users showed that the acquisition speed of web sentences becomes 2.6 times faster than ordinal screen reader.Research results regarding "the captioning system" are as followings.(1)We investigated how a speaker's facial image and lip-movement (the non-verbal information) were useful for the comprehension of the sentenses(2)Experimental results show that the lip-movement was more effective than facial expression and about 2 sec delay of the lip-movement display from the captions was suitable for the fascilitation of the comprehension.(3)By combining this captioning system with the non-verbal information, the comprehension was proved apparently fascilitated through about 30 times trials at international conferences.Non-verbal information ; the tactile stimulation in TAJODA and the lip-movement in the captioning system was apparently effective to fascilitate the comprehension of contents. Furthermore, we could determin the optimal parameters of the display timing and the display method of the non-verbal information in order to improve the information assistive tools for the sensory disabled. Less