Perception and cognition of musical timbre

音色是研究最少的听觉属性之一，但它在听觉的许多方面都起着至关重要的作用。音色是指声音的许多属性，而不是音高、持续时间、响度和空间位置。对它的感知表征进行建模的尝试相当初级。在世界其他地方，还没有一个综合性的项目涉及音色表征的许多依赖于任务的方面，并将它们与一般的听觉感知，特别是音乐感知联系起来。该计划将扩展与音色的感知维度相关的标量声学参数的工作，以随时间变化的参数和更高维的表示，如听觉皮层神经元建模的spectrotemporal感受野的心理物理学。我将面对这些不同的感觉表征和特征提取模型，以评估它们解释与相似性感知、感知融合、听觉流形成、听觉显著性、声源识别和记忆效应相关的感知结果的能力。我将确定感知表征如何捕捉音色的方式，音高，并在乐器的声音演奏努力的协变，并将研究他们的贡献，通过收购声源的心理模型识别和识别。我将定义并发声音的听觉表征之间的关系如何影响它们的感知融合，并确定由此产生的混合音色质量。我将研究影响听觉流分离的音色表征的各个方面，以及通过内在显著性音色特性或外在语境显著性来捕获听者的注意力。最后，我将研究音色在听觉记忆中的编码方式，并将研究记忆痕迹和声源的感知模型对音色相异结构的影响。这项研究计划将把我们对音色处理的理解提升到一个新的水平，通过引入一个涉及时变特性的定量框架，通过系统地评估和比较表征模型的准确性，概括性和简约性，在广泛的数据集上，通过表征不同感知任务下特征表征的延展性，并通过证明音色有助于注意力，学习和记忆。该计划将提供一个独特的培训环境，用于获得行为测试技术，实验设计，复杂的数据分析和统计，听觉过程的数学建模和数字信号处理方面的技能。它将对各种应用领域产生影响，如听觉假体，音乐推荐和计算机辅助编排。

Timbre is one of the least studied auditory attributes, and yet it plays a crucial role in many aspects of audition. Timbre refers to many attributes of sound other than pitch, duration, loudness, and spatial position. Attempts to model its perceptual representation have been fairly rudimentary. No comprehensive program elsewhere in the world addresses the many task-dependent aspects of timbre representation and relates them to auditory perception in general and music perception in particular. This program will extend work on scalar acoustic parameters that correlate with the perceptual dimensions of timbre to the psychophysics of time-varying parameters and higher-dimensional representations such as spectrotemporal receptive fields modeled on auditory cortical neurons. I will confront these different models of sensory representation and feature extraction to assess their ability to account for perceptual results related to similarity perception, perceptual fusion, auditory stream formation, auditory salience, recognition of sound sources, and memory effects. I will determine how perceptual representations capture the way timbre, pitch, and playing effort covary in instrument sounds and will study their contribution to recognition and identification through the acquisition of mental models of sound sources. I will define how the relations between auditory representations of concurrent sounds condition their perceptual fusion and determine the resulting blended timbral qualities. I will study the aspects of timbre representation that affect auditory stream segregation and that capture listeners’ attention by intrinsic salient timbral properties or extrinsic contextual salience. Finally, I will investigate the ways in which timbre is encoded in auditory memory and will study the impact of memory traces and perceptual models of sound sources on timbre dissimilarity structures. This research program will take our understanding of timbre processing to the next level by introducing a quantitative framework involving time-varying properties, by systematically evaluating and comparing representation models in terms of accuracy, generalizability and parsimony on a wide range of data sets, by characterizing the malleability of feature representations under different perceptual tasks, and by demonstrating that timbre contributes to attention, learning and memory. The program will provide a unique training environment for the acquisition of skills in behavioural testing techniques, experimental design, complex data analysis and statistics, mathematical modeling of auditory processes, and digital signal processing. It will have an impact on a variety of application domains such as auditory prostheses, music recommendation, and computer-aided orchestration.