ITR/AITS: Customizable Audio User Interfaces for the Visually Impaired and the Sighted

尽管我们大脑的大部分致力于处理声音线索，声音在我们与世界的交互方式中起着重要作用，但这个丰富的渠道尚未被广泛地用于显示信息。接收到的声波通过神经处理形成具有许多感知维度听觉特性的物体的机制，包括三个对应于源位置（范围，方位角，高度）和三个对应于源的质量（音色，音高和强度）的机制，正在开始被理解。在过去的十年中，在理解声音线索产生的机制，生物系统如何进行转导和神经处理以从声音中提取相关特征，以及我们在声音场景中感知和组织物体的方式方面取得了重大进展。我们的目标是利用这种理解，并揭示控制包含多个信息和特征丰富的声音对象的人工声音场景的计算机渲染的科学原理。我们将通过为视障人士和视力正常的人创建听觉用户界面来测试、使用和扩展这些知识。这项工作旨在开发界面并回答基本问题，例如：是否有可能将“X”有效地映射到虚拟听觉空间的听觉轴上？这里的“X”可以是图像（例如，一张脸）、地图、表格数据、不确定数据或临时变化的数据。是否有神经关联可以引导自然映射到声音线索？我们的感知对渲染硬件有什么限制？有多重要？

Although large parts of our brains are devoted to the processing of sound cues and sound plays an important role in the way we interface with the world, this rich channel has not been extensively exploited for displaying information. The mechanisms by which received sound waves are processed neurally to form objects with auditory properties in many perceptual dimensions, including three corresponding to the source location (range, azimuth, elevation) and three to qualities ascribed to the source (timbre, pitch and intensity), are beginning to be understood. There has been significant progress over the last decade in understanding the mechanisms by which acoustical cues arise and how the biological system performs transduction and neural processing to extract relevant features from sound, and in the way we perceive and organize objects in acoustical scenes. Our goal is to exploit this understanding, and uncover the scientific principles that govern the computerized rendering of artificial sound scenes containing multiple sound objects that are information and feature rich. We will test, use and extend this knowledge by creating auditory user interfaces for the visually impaired and the sighted. The work aims both at developing interfaces and answering fundamental questions such as: Is it possible to usefully map "X" to the auditory axes of a virtual auditory space? Here "X" could be an image (e.g., a face), a map, tabular data, uncertain data, or temporally varying data. Are there neural correlates that can guide natural mappings to acoustic cues? What limitations does our perception place on rendering hardware? How important is