A Study of Barrier-free Communication Systems Using Virtual Reality Technology for Deaf People

在本研究项目中，我们开发了三种类型的听力残疾人无障碍沟通系统。这些系统采用了最新的技术，如虚拟现实中的传感器显示器和人机界面技术中的可穿戴PC。本计画的成果将在以下的段落中详细叙述。1.一个可携式的自动笔记系统已经被设计出来，让聋人与听力正常的人可以面对面的交流。该系统主要由一个轻型的光学透视头盔显示器，可穿戴PC和语音识别软件。为了确认VR显示系统的安全裕度，对人的影响进行了神经眼科检查。结果发现，长时间的VR显示器视觉负荷会影响健康受试者的眼调节和瞳孔运动。2.我们的紧凑型触觉辅助器采用多通道振动触觉显示。为了帮助聋人进行语言阅读，这种触觉交流装置将 关于我们 将声音信息转化为指尖上的振动触觉刺激。我们在这里调查的生物力学特性的指垫皮肤产生足够的触觉刺激的语音通信和设计的宽带振动触觉显示器组成的小型压电双晶片致动器。从心理物理学实验的结果，有人建议，我们的触觉援助应允许目前的语音质量的发言人的指尖。3.实时字幕系统的开发，公众沟通的会议或讲座的学校。该系统由LCD投影仪代替上述笔记系统的MHD、同时重峰操作器以更灵活地响应各种发言者、以及用于将发言者的图像和字幕叠加在大屏幕上的装置组成。此外，我们研究了最佳的时空参数叠加的图像的扬声器和字幕，以提高沟通的准确性聋人用户。这一系统现已在一次国际会议和日本一些大学的讲座上提供。少

In this research project, we developed three types of barrier-free communication system for people with hearing disability. These systems employ the recent technology such as a sensory display in virtual reality and a wearable PC in human interface technology. Specifically the results of this project are described in the following paragraphs.1.A portable automatic note-taking system has been designed to allow face-to-face communication between deaf and hearing people. This system is mainly composed of a lightweight optical see-through HMD, a wearable PC and speech recognition software. To confirm a margin of safety of the VR display system, human influences were examined neuro-ophthalmologically. It was found that visual load by the VR display for a long time affected ocular accommodation and pupillary movement in healthy subjects.2.Our compact tactile aid utilizes a multi-channel vibrotactile display. To assist deaf people at speech reading, this tactile communication device converts … More voice information into vibrotactile stimulation on a fingertip. We investigate here biomechanical characteristics of fingerpad skin to generate adequate tactile stimulus for voice communication and design a broadband vibrotactile display composed of small piezoelectric bimorph actuators. From the results of psychophysical experiments, it was suggested that our tactile aid should allow to present voice quality of a speaker on the fingertip.3.A real-time captioning system was developed for public communication at a meeting or the lectures of a school. This system consists of a LCD projector instead of the MHD of the above note-taking system, a simultaneous respeaking operator to respond more flexibly to various speakers, and a device for superimposing the images of a speaker and a caption on a large screen. Furthermore we examined the optimal spatiotemporal parameters of superimposing the images of a speaker and a caption to improve communication accuracy for deaf users. This system is now available at an international meeting and the lectures of some universities in Japan. Less