EAGER: Spatial Audio Data Immersive Experience (SADIE)

虽然最近有很多人对可视化来支持数据分析感兴趣，但声化——将非听觉信息渲染为声音——代表了一个相对未被探索但丰富的空间，可以映射到许多数据分析问题，特别是当数据具有自然的空间和时间元素时。与基于耳机的声音处理方法不同，该项目将探索“外心”声音环境的潜力，这种环境完全包围了用户，并允许他们通过在空间中移动来与声音数据交互。假设是，与现有的数据分析方法相比，空间数据与空间表示的耦合，通过运动与数据交互的自然性，利用人类听到模式并在3D中定位它们的能力，以及避免基于耳机的超声策略引入的伪像，都将帮助人们感知数据中的模式和因果关系。为了验证这一点，该团队将开发一套原语，用于将时空数据映射到声音参数（如音量、音调和频谱过滤）。他们将通过一系列日益复杂的数据分析实验，包括地球空间科学领域的具体分析任务，来完善这些原语。如果成功，这项工作可能会在各种应用中产生影响，从增强可视化到开发更好的虚拟现实系统，同时在从音乐到计算再到物理科学的科学社区之间建立跨学科的桥梁。该项目将使用一个身临其境的声音工作室进行开发，该工作室包括运动跟踪功能和高密度扬声器阵列，由团队开发的算法和开源声音库驱动，以支持具体的、丰富的声学数据探索，这些数据不受基于耳机的策略（如头部相关传递函数）引入的音频偏差的影响。针对单个数据流的具体声化策略将基于前面描述的原语，重点关注谱丰富的声音，这些声音更容易被人们定位。表示多个数据流的策略将包括分层多个非屏蔽声音和组合流来调制相同声音的不同方面（例如，音高和音量）。为了开发和验证这些策略，团队将进行一系列实验，逐步增加分析任务的复杂性：从感知和解释单个数据原语的基本能力，到感知和推断多个数据流之间的关系，再到测量受试者在一系列地理空间模型场景中感知多个数据流之间已知原因的能力。在这些研究中，团队将改变数据中关系的强度，声音参数操作的大小，以及不同声音和参数化的配对，以确定声学策略的感知属性和局限性（在某些方面类似于基于感知的可视化基础）；他们还将比较参与者在使用外中心环境和以耳机为基础的自我中心环境作为对照时的分析表现和定性反应。

Although there has been much recent interest in visualization to support data analysis, sonification -- the rendering of non-auditory information as sound -- represents a relatively unexplored but rich space that could map onto many data analysis problems, especially when the data has a natural spatial and temporal element. In contrast to headphone-based sonification approaches, this project will explore the potential of "exocentric" sound environments that completely encompass the user and allow them to interact with the sonified data by moving in space. The hypothesis is that compared to existing methods of data analysis, the coupling of spatial data with spatial representations, the naturalness of interacting with the data through motion, the leveraging of humans' ability to hear patterns and localize them in 3D, and the avoidance of artifacts introduced by headphone-based sonification strategies will all help people perceive patterns and causal relationships in data. To test this, the team will develop a set of primitives for mapping spatio-temporal data to sound parameters such as volume, pitch, and spectral filtering. They will refine these primitives through a series of increasingly complex data analysis experiments, including specific analysis tasks in the domain of geospace science. If successful, the work could have implications in a variety of applications, from enhancing visualizations to developing better virtual reality systems, while developing interdisciplinary bridges between scientific communities from music to computing to the physical sciences.The project will be developed using an immersive sound studio that includes motion tracking capabilities and a high-density loudspeaker array, driven by algorithms and open source sound libraries developed by the team to support embodied, rich exploration of sonified data that is not subject to audio deviations introduced by headphone-based strategies such as Head Related Transfer Functions. The specific sonification strategies for individual data streams will be based on the primitives described earlier, focusing on sounds rich in spectra that are easier for people to localize. Strategies for representing multiple data streams will include layering multiple non-masking sounds and combining streams to modulate different aspects of the same sound (e.g., pitch and volume). To develop and validate the strategies, the team will conduct a series of experiments that gradually increase the complexity of the analysis tasks: from basic ability to perceive and interpret single data primitives, to perceiving and inferring relationships between multiple data streams, to measuring subjects' ability to perceive known causes between multiple data streams in a series of geospatial model scenarios. In these studies the team will vary the strength of relationships in the data, the size of the parameter manipulations of sounds, and the pairing of different sounds and parameterizations in order to determine perceptual properties and limitations of sonficiation strategies (similar in some ways to perception-based foundations of visualization); they will also compare both analysis performance and qualitative reactions of participants using both the exocentric environment and a headphone-based egocentric environment as a control.