Development of Sonification Design Theory: Metaphors, Mappings, Holistic Sound Design, and Data Specific Sonification

数据的听觉表示，或发声，有望在时间和高维数据的表示中、在眼睛忙碌的数据监控任务中、在需要识别数据集中的模式的任务中、在高压力或关键条件下(其中跨模式关联将是有价值的)、以及在显示用户是视力残疾的学生或研究人员的情况下获得收益。然而，确定将数据维度(如温度或压力)映射到听觉显示维度(如音调或节奏)的最佳方法所做的工作很少。我们对声音产生的隐喻联想也知之甚少，也不知道如何利用这些内涵来设计出易于学习和直观使用的有效、引人注目和令人愉快的发音。这项研究试图发现用于科学发声的最佳数据-显示映射，并调查这些最佳映射是否在应用领域内和/或跨应用领域变化。为了实现这一目标，PI和他的团队将开发一个跨平台的研究环境，其中包括复杂的声音合成和数据收集能力。一系列实验将调查：(A)数据到声音映射中的听者偏好和群体刻板印象；(B)不同科学领域的映射有何不同；以及(C)数据发声对图形解释和数据挖掘的有效性。PI将开发一个跨平台发声软件工具，以创建适合研究领域、用户特征和要探索的数据类型的听觉显示。这一工具将促进许多学科的数据分析，并为视力残疾的科学家和学生提供直接进入科学界的途径。

Auditory representation of data, or sonification, promises gains in the representation of temporal and high-dimensional data, in data monitoring tasks where the eyes are busy, in tasks requiring recognition of patterns in a data set, in high-stress or critical conditions where cross-modal correlations would be of value, and where the display users are students or researchers with visual disabilities. However, very little work has been done to determine the best ways to map data dimensions, such as temperature or pressure, onto auditory display dimensions, such as pitch or tempo. We also know little about the metaphorical associations that sounds create, and how to utilize these connotations to design effective, compelling, and pleasant sonifications which are easy to learn and intuitive to use. This research seeks to discover the optimal data-to-display mappings for use in scientific sonification, and to investigate whether these optimal mappings vary within and/or across fields of application. To accomplish this, the PI and his team will develop a cross-platform research environment that incorporates sophisticated sound synthesis and data collection capabilities. A series of experiments will investigate: (a) listener preferences and population stereotypes in data-to-sound mappings; (b) how mappings differ across various scientific areas; and (c) the effectiveness of data sonification for graphical interpretations and data mining. The PI will develop a cross-platform sonification software tool to create auditory displays suitable for the domain of study, the characteristics of the user, and the type of data to be explored. This tool will facilitate data analysis in a number of disciplines, and provide visually-disabled scientists and students immediate inroads into the world of science.