ITR/AITS: Customizable Audio User Interfaces for the Visually Impaired and the Sighted

尽管我们大脑的很大一部分用于处理声音提示，声音在我们与世界交互的方式中扮演着重要的角色，但这一丰富的渠道还没有被广泛地利用来显示信息。接收到的声波被神经处理以形成具有许多感知维度的听觉特性的物体的机制开始被理解，其中包括三个对应于源位置(距离、方位、仰角)和三个对应于源的特性(音色、音调和强度)。在过去的十年里，在理解声学线索产生的机制，生物系统如何执行转导和神经处理以从声音中提取相关特征，以及在我们感知和组织声学场景中的对象的方式方面，已经取得了重大进展。我们的目标是利用这种理解，并揭示管理人工声音场景的计算机渲染的科学原理，这些人工声音场景包含多个信息丰富且功能丰富的声音对象。我们将通过为视障人士和视障人士创建听觉用户界面来测试、使用和扩展这些知识。这项工作既旨在开发界面，也旨在回答一些基本问题，例如：是否有可能有效地将“X”映射到虚拟听觉空间的听觉轴线上？这里，X可以是图像(例如，人脸)、地图、表格数据、不确定数据或时间变化数据。有没有神经关联可以引导自然映射到声学线索？我们对渲染硬件的认知有哪些限制？有多重要？

Although large parts of our brains are devoted to the processing of sound cues and sound plays an important role in the way we interface with the world, this rich channel has not been extensively exploited for displaying information. The mechanisms by which received sound waves are processed neurally to form objects with auditory properties in many perceptual dimensions, including three corresponding to the source location (range, azimuth, elevation) and three to qualities ascribed to the source (timbre, pitch and intensity), are beginning to be understood. There has been significant progress over the last decade in understanding the mechanisms by which acoustical cues arise and how the biological system performs transduction and neural processing to extract relevant features from sound, and in the way we perceive and organize objects in acoustical scenes. Our goal is to exploit this understanding, and uncover the scientific principles that govern the computerized rendering of artificial sound scenes containing multiple sound objects that are information and feature rich. We will test, use and extend this knowledge by creating auditory user interfaces for the visually impaired and the sighted. The work aims both at developing interfaces and answering fundamental questions such as: Is it possible to usefully map "X" to the auditory axes of a virtual auditory space? Here "X" could be an image (e.g., a face), a map, tabular data, uncertain data, or temporally varying data. Are there neural correlates that can guide natural mappings to acoustic cues? What limitations does our perception place on rendering hardware? How important is