A Study of Barrier-free Communication Systems Using Virtual Reality Technology for Deaf People

在这个研究项目中，我们为听障人士开发了三种无障碍交流系统。这些系统采用了最近的技术，如虚拟现实中的感官显示器和人机界面技术中的可穿戴PC。具体来说，这个项目的结果将在以下段落中描述。一种便携式自动笔记系统被设计出来，使聋哑人和听力正常的人能够面对面交流。该系统主要由轻型光学透明头戴式显示器、可穿戴PC和语音识别软件组成。为了确定VR显示系统的安全范围，我们从神经眼科角度检查了人的影响。研究发现，长时间VR显示的视觉负荷会影响健康受试者的眼部调节和瞳孔运动。我们的紧凑型触觉辅助设备采用多通道振动触觉显示。为了帮助聋哑人阅读语言，这种触觉交流设备将更多的语音信息转化为指尖上的振动触觉刺激。本文研究指垫皮肤的生物力学特性，为语音交流提供足够的触觉刺激，并设计了一种由小型压电双晶片致动器组成的宽带振动触觉显示器。从心理物理实验的结果来看，有人建议我们的触觉辅助设备应该允许在指尖上呈现说话人的声音质量。开发了实时字幕系统，用于在会议或学校讲座中进行公共交流。该系统由LCD投影仪代替上述笔记系统的MHD，同时对各种扬声器做出更灵活反应的同声传译操作员，以及在大屏幕上叠加扬声器图像和字幕的装置组成。此外，我们还研究了说话人和字幕叠加的最佳时空参数，以提高聋人用户的沟通准确性。这个系统现在可以在国际会议和日本一些大学的讲座中使用。少

In this research project, we developed three types of barrier-free communication system for people with hearing disability. These systems employ the recent technology such as a sensory display in virtual reality and a wearable PC in human interface technology. Specifically the results of this project are described in the following paragraphs.1.A portable automatic note-taking system has been designed to allow face-to-face communication between deaf and hearing people. This system is mainly composed of a lightweight optical see-through HMD, a wearable PC and speech recognition software. To confirm a margin of safety of the VR display system, human influences were examined neuro-ophthalmologically. It was found that visual load by the VR display for a long time affected ocular accommodation and pupillary movement in healthy subjects.2.Our compact tactile aid utilizes a multi-channel vibrotactile display. To assist deaf people at speech reading, this tactile communication device converts … More voice information into vibrotactile stimulation on a fingertip. We investigate here biomechanical characteristics of fingerpad skin to generate adequate tactile stimulus for voice communication and design a broadband vibrotactile display composed of small piezoelectric bimorph actuators. From the results of psychophysical experiments, it was suggested that our tactile aid should allow to present voice quality of a speaker on the fingertip.3.A real-time captioning system was developed for public communication at a meeting or the lectures of a school. This system consists of a LCD projector instead of the MHD of the above note-taking system, a simultaneous respeaking operator to respond more flexibly to various speakers, and a device for superimposing the images of a speaker and a caption on a large screen. Furthermore we examined the optimal spatiotemporal parameters of superimposing the images of a speaker and a caption to improve communication accuracy for deaf users. This system is now available at an international meeting and the lectures of some universities in Japan. Less