A Study on Usages of Non-verbal Information for a Design of Information Assistive Tools

我们研究了如何利用非语言信息来改进我们为视障人士设计的两个信息获取工具。1.其中一个工具是一个新的屏幕阅读器系统，它通过使用触觉微动拨盘(TAJoda)将屏幕上的文本转换为合成语音，同时在指尖表面显示丰富的文本信息，即非语言信息。另一种工具是一种新型的助听器(字幕系统)，它可以自动地将语音转换成文本，同时为获得的听障人士显示说话人的嘴唇运动和面部表情，即非言语信息。关于屏幕阅读器的研究结果如下：(1)合成语音的速度应该比正常说话速度快3倍，因为我们的研究结果表明，盲人的最大收听速度是…的2.6倍(2)传输丰富文本的最佳数量为7个；即字体大小(大于或小于标准)、彩色字母、粗体、括号(左或右)、(3)8个盲人用户的实验结果表明，网络句子的获取速度比普通屏幕阅读器快2.6倍。(1)考察了说话者的面部图像和嘴唇运动(非语言信息)对句子理解的作用。(2)实验结果表明，嘴唇运动比面部表情更有效，字幕中嘴唇运动的显示延迟约2秒，适合于理解。(3)将该字幕系统与非语言信息相结合，通过在国际会议上进行大约30次的试验，这种理解被证明是明显的。TAJoda中的触觉刺激和字幕系统中的唇动对内容的理解有明显的效果。此外，我们还可以确定非语言信息的最佳显示时机和显示方法的参数，以改进感官障碍者的信息辅助工具。较少

We investigated how non-verbal information should be utilized in order to improve two information acquisition tools which we designed for the visually and auditory impaired.1.One of the tools is a new screen reader system which converts texts on the screen into synthesized speech while displaying rich text information, i.e. non-verbal information, onto a finger-tip surface by using a tactile jog-dial (TAJODA). This is a supporting tool of a web-information acquisition for visually impaired people.2.The other tool is a new hearing assistive device (captioning system) which automatically converts speech into texts while displaying speaker's lip-movement and facial expression, i.e. non-verbal information, for the acquired hearing impaired.Research results regarding "the screen reader" are as followings.(1)The speed of synthesized speech should be changed up to 3 times faster than normal speaking rate since our findings showed the maximum listening speeds for the blind is 2.6 times faster … More than the normal persons.(2)The optimal number of rich texts to transmit is 7;i.e. font sizes (larger or smaller than the standard), colored letter, bold, parentheses (left or right), and paragraph.(3)Experimental results by 8 blind users showed that the acquisition speed of web sentences becomes 2.6 times faster than ordinal screen reader.Research results regarding "the captioning system" are as followings.(1)We investigated how a speaker's facial image and lip-movement (the non-verbal information) were useful for the comprehension of the sentenses(2)Experimental results show that the lip-movement was more effective than facial expression and about 2 sec delay of the lip-movement display from the captions was suitable for the fascilitation of the comprehension.(3)By combining this captioning system with the non-verbal information, the comprehension was proved apparently fascilitated through about 30 times trials at international conferences.Non-verbal information ; the tactile stimulation in TAJODA and the lip-movement in the captioning system was apparently effective to fascilitate the comprehension of contents. Furthermore, we could determin the optimal parameters of the display timing and the display method of the non-verbal information in order to improve the information assistive tools for the sensory disabled. Less