Development of Sonification Design Theory: Metaphors, Mappings, Holistic Sound Design, and Data Specific Sonification

数据的听觉表示或声音化有望在时间和高维数据的表示、眼睛忙碌的数据监测任务、需要识别数据集中模式的任务、跨模态关联有价值的高压力或关键条件、以及显示用户是有视觉障碍的学生或研究人员的情况下获得收益。然而，在确定将数据维度（如温度或压力）映射到听觉显示维度（如音高或节奏）的最佳方法方面，人们做的工作很少。我们也不太了解声音所创造的隐喻关联，以及如何利用这些内涵来设计有效、引人注目和令人愉快的声音，这些声音易于学习和直观使用。本研究旨在发现用于科学超声的最佳数据显示映射，并调查这些最佳映射是否在应用领域内和/或跨领域变化。为了实现这一目标，PI和他的团队将开发一个跨平台的研究环境，包括复杂的声音合成和数据收集功能。一系列实验将调查：(A)数据到声音映射中的听众偏好和人口刻板印象；(b)不同科学领域的映射如何不同；(c)图形解释和数据挖掘的数据声化的有效性。PI将开发一种跨平台的声音化软件工具，以创建适合研究领域、用户特征和要探索的数据类型的听觉显示。该工具将促进许多学科的数据分析，并为视障科学家和学生提供立即进入科学世界的途径。

Auditory representation of data, or sonification, promises gains in the representation of temporal and high-dimensional data, in data monitoring tasks where the eyes are busy, in tasks requiring recognition of patterns in a data set, in high-stress or critical conditions where cross-modal correlations would be of value, and where the display users are students or researchers with visual disabilities. However, very little work has been done to determine the best ways to map data dimensions, such as temperature or pressure, onto auditory display dimensions, such as pitch or tempo. We also know little about the metaphorical associations that sounds create, and how to utilize these connotations to design effective, compelling, and pleasant sonifications which are easy to learn and intuitive to use. This research seeks to discover the optimal data-to-display mappings for use in scientific sonification, and to investigate whether these optimal mappings vary within and/or across fields of application. To accomplish this, the PI and his team will develop a cross-platform research environment that incorporates sophisticated sound synthesis and data collection capabilities. A series of experiments will investigate: (a) listener preferences and population stereotypes in data-to-sound mappings; (b) how mappings differ across various scientific areas; and (c) the effectiveness of data sonification for graphical interpretations and data mining. The PI will develop a cross-platform sonification software tool to create auditory displays suitable for the domain of study, the characteristics of the user, and the type of data to be explored. This tool will facilitate data analysis in a number of disciplines, and provide visually-disabled scientists and students immediate inroads into the world of science.