On Human Factor in a Virtual Space Integrating Visual Auditory and tactile Modalities

1.在过去十年中，从硬件和软件的角度来看，实现虚拟空间或网络空间的技术都得到了改进。基于这样的环境，作者试图建立关于整合视觉、听觉和触觉通道的理论方法论基础。在这篇报告中，我们研究了人类的感觉替代方式，即视觉通道如何被另一种通道(如听觉或触觉通道)替代为空间知觉。也就是说，通过提出使用3-D听觉显示和3-D非语言语音输入的3-D(以下简称3-D)交互策略，作者通过实验证明，通过使用适当的听觉接口和短期的练习，可以在没有任何视觉信息的情况下充分地操纵3-D对象。通过这样做，不仅确认了听觉通道的意义，而且是ma…更多的De，但也暗示了仅靠听觉媒介进行虚拟空间操纵的可能性。其次，作者对触觉媒介的感觉替代功能进行了类似的研究。也就是说，通过提出触觉反馈机制来显示安装在用户身上的超声波传感器计算出的用户周围环境的深度信息，研究了在没有视觉通道的情况下通过触觉通道进行空间感知和空间操纵的可能性。本研究通过心理学实验和任务绩效实验，证实听觉通道和触觉通道都能较好地替代视觉通道在空间知觉方面的作用，这将为残障人士提供一种新型的人机交互界面。此外，还给出了几个集成这些模式的应用实例，以实现高效的网络空间，以讨论多模式接口的重要性。我们未来的工作包括从实现虚拟空间的角度研究人类的环境适应性，包括分布式虚拟环境(DVE)和遥虚拟(TVR)的时间人因素。较少

1. During the last decade the technology for implementation of a virtual space or a cyberspace has been improved from both of the hardware and the software viewpoints. Based on such environment the authors have attempted to establish the theoretical methodological basics regarding the way of integrating the visual, auditory and tactile modalities. In this report, on the preparation for the objective, we have researched the way of human's sensory substitution of how the visual modality for spatial perception is substituted by another modality (e.g.auditory or tactile modality). That is, by proposing the three dimensional (3-D hereafter) interaction strategy using 3-D auditory display and 3-D nonverbal voice input, the authors have shown experimentally that manipulating 3-D object might be sufficiently performed without any visual information by utilizing suitable auditory interface with a short term practice. By doing so, not only confirmation of the significance auditory modality is ma … More de, but also the possibility of virtual spatial manipulation just by auditory media is implied.2. Secondly the authors have attempted the similar research on the sensory substitution function by tactile media. That is, by proposing tactile feedback mechanism for display of depth information of the surroundings of a user calculated by the ultrasonic sensor mounted on user' body, the possibility of spatial perception and spatial manipulation by tactile modality in case of the absence of visual modality is researched. As a result, the authors have confirmed that either auditory modality or tactile modality might be able to substitute satisfactorily the visual modality in terms of spatial perception by psychological experiments as well as task performance experiments, which would be a significant contribution to a new type of man machine interface for disabled or handicapped person.3. Moreover, several examples of application which integrates these modalities are shown for implementing efficient cyber space, to discuss the importance of multi-modal interface. Our future works include the research of human's environmental adaptability in terms of realization for reality of a virtual space, including temporal human factor for distributed virtual environment (DVE) and televirtuality (TVR). Less