On Human Factor in a Virtual Space Integrating Visual Auditory and tactile Modalities

1. 在过去的十年中，实现虚拟空间或网络空间的技术从硬件和软件的角度都得到了改进。基于这样的环境，笔者试图建立视觉、听觉和触觉相结合的理论方法基础。在这篇报道中，在为目标做准备的基础上，我们研究了人类的感官替代方式，即空间感知的视觉形态如何被另一种形态（如听觉或触觉形态）所替代。也就是说，通过提出三维听觉显示和三维非语言语音输入的三维交互策略，作者通过实验证明，在没有任何视觉信息的情况下，通过短期的练习，利用合适的听觉界面，可以充分完成对三维物体的操作。这不仅证实了听觉模态的重要性，而且暗示了仅仅通过听觉媒介操纵虚拟空间的可能性。其次，作者对触觉媒介的感觉替代功能进行了类似的研究。即通过提出由安装在用户身体上的超声波传感器计算的用户周围环境深度信息的触觉反馈显示机制，研究在没有视觉模态的情况下，通过触觉模态进行空间感知和空间操纵的可能性。因此，作者通过心理实验和任务执行实验证实，听觉或触觉方式都可以在空间感知方面令人满意地替代视觉方式，这将为残疾人或残疾人的新型人机界面做出重要贡献。此外，还展示了集成这些模式的几个应用实例，以实现高效的网络空间，讨论多模式接口的重要性。我们未来的工作包括从虚拟空间的现实性实现角度研究人的环境适应性，包括分布式虚拟环境（DVE）和虚拟环境（TVR）的时间人为因素。少

1. During the last decade the technology for implementation of a virtual space or a cyberspace has been improved from both of the hardware and the software viewpoints. Based on such environment the authors have attempted to establish the theoretical methodological basics regarding the way of integrating the visual, auditory and tactile modalities. In this report, on the preparation for the objective, we have researched the way of human's sensory substitution of how the visual modality for spatial perception is substituted by another modality (e.g.auditory or tactile modality). That is, by proposing the three dimensional (3-D hereafter) interaction strategy using 3-D auditory display and 3-D nonverbal voice input, the authors have shown experimentally that manipulating 3-D object might be sufficiently performed without any visual information by utilizing suitable auditory interface with a short term practice. By doing so, not only confirmation of the significance auditory modality is ma … More de, but also the possibility of virtual spatial manipulation just by auditory media is implied.2. Secondly the authors have attempted the similar research on the sensory substitution function by tactile media. That is, by proposing tactile feedback mechanism for display of depth information of the surroundings of a user calculated by the ultrasonic sensor mounted on user' body, the possibility of spatial perception and spatial manipulation by tactile modality in case of the absence of visual modality is researched. As a result, the authors have confirmed that either auditory modality or tactile modality might be able to substitute satisfactorily the visual modality in terms of spatial perception by psychological experiments as well as task performance experiments, which would be a significant contribution to a new type of man machine interface for disabled or handicapped person.3. Moreover, several examples of application which integrates these modalities are shown for implementing efficient cyber space, to discuss the importance of multi-modal interface. Our future works include the research of human's environmental adaptability in terms of realization for reality of a virtual space, including temporal human factor for distributed virtual environment (DVE) and televirtuality (TVR). Less