Seeing through the Ears

点击翻译按钮获取中文摘要

This is a revised submission of a proposal to use psychophysics and brain imaging to investigate the brain activity associated with the processing of environmental visual information presented to the auditory system in blind and sighted individuals. Peter Meijer developed the first camera-based auditory display to preserve a significant amount of environmental visual information in sound, while taking into account the major constraints imposed by both the human auditory system and the frequency-time uncertainty relation (The vOICe, U.S. Patent 5,097,326). The vOICe results in a vision substitution tool for the blind providing a means of presenting live image-to-sound converted scenery and permits the study of cross-modal plasticity. Three groups of subjects will be studied: sighted controls, blind, and sighted-blindfolded subjects,. We will compare results across and within groups before and after training in the use of the image-to-sound conversion system. Training will be conducted daily for four weeks and will combine formal instruction using controlled exercises and more naturalistic, immersive use of the vOICe by the subject while navigating the environment in the laboratory. Three main tasks of decoding and processing visual information by the auditory system will be studied to address: (1) Topographic organization, (2) Mechanisms of object identification, and (3) Object constancy. Behavioral measures, functional magnetic resonance imaging (fMRI), and transcranial magnetic stimulation (TMS) will be used. We expect to find significant differences between the groups of subjects in the behavioral measures and brain activity associated with the processing of image-to-sound converted information. This study will provide critical insights into the mechanisms underlying cross-modal plasticity and processing of environmental spatial visual information presented via the auditory system in blind, as well as about metamodal object and space representation. Image-to-sound conversion may eventually become a suitable prosthesis for the blind.