Seeing through the Ears

这是一份关于使用心理物理学和脑成像来研究与处理呈现给听觉系统的环境视觉信息相关的大脑活动的提案的修订稿。 在盲人和有视力的人身上。Peter Meijer开发了第一个基于相机的听觉显示，以保留声音中大量的环境视觉信息，同时考虑到主要的 人类听觉系统和频率-时间不确定性关系造成的限制(语音，美国专利5,097,326)。该声音导致盲人的视觉替代工具提供 呈现现场图像-声音转换场景的方法，并允许研究跨模式可塑性。将对三组受试者进行研究：有视力的对照组、盲人和失明受试者。我们会 在使用图像到声音转换系统的培训前后，比较组内和组内的结果。培训将持续四周，每天进行，并将结合使用受控练习的正式指导，以及受试者在实验室导航环境时更自然、更身临其境的声音使用。我们将研究听觉系统对视觉信息进行解码和处理的三个主要任务：(1)地形组织；(2)物体机制 识别性，以及(3)对象的恒定性。将使用行为测量、功能磁共振成像(FMRI)和经颅磁刺激(TMS)。我们预计会发现显著的差异 两组受试者之间的行为测量和大脑活动与图像到声音转换信息的处理有关。这项研究将对这一机制提供关键的见解。 盲人通过听觉系统呈现的环境空间视觉信息的跨通道可塑性和加工，以及元通道物体和空间表征。图像到声音的转换最终可能成为适合盲人的假体。

This is a revised submission of a proposal to use psychophysics and brain imaging to investigate the brain activity associated with the processing of environmental visual information presented to the auditory system in blind and sighted individuals. Peter Meijer developed the first camera-based auditory display to preserve a significant amount of environmental visual information in sound, while taking into account the major constraints imposed by both the human auditory system and the frequency-time uncertainty relation (The vOICe, U.S. Patent 5,097,326). The vOICe results in a vision substitution tool for the blind providing a means of presenting live image-to-sound converted scenery and permits the study of cross-modal plasticity. Three groups of subjects will be studied: sighted controls, blind, and sighted-blindfolded subjects,. We will compare results across and within groups before and after training in the use of the image-to-sound conversion system. Training will be conducted daily for four weeks and will combine formal instruction using controlled exercises and more naturalistic, immersive use of the vOICe by the subject while navigating the environment in the laboratory. Three main tasks of decoding and processing visual information by the auditory system will be studied to address: (1) Topographic organization, (2) Mechanisms of object identification, and (3) Object constancy. Behavioral measures, functional magnetic resonance imaging (fMRI), and transcranial magnetic stimulation (TMS) will be used. We expect to find significant differences between the groups of subjects in the behavioral measures and brain activity associated with the processing of image-to-sound converted information. This study will provide critical insights into the mechanisms underlying cross-modal plasticity and processing of environmental spatial visual information presented via the auditory system in blind, as well as about metamodal object and space representation. Image-to-sound conversion may eventually become a suitable prosthesis for the blind.