Perception and cognition of musical timbre

点击翻译按钮获取中文摘要

Timbre is one of the least studied auditory attributes, and yet it plays a crucial role in many aspects of audition. Timbre refers to many attributes of sound other than pitch, duration, loudness, and spatial position. Attempts to model its perceptual representation have been fairly rudimentary. No comprehensive program elsewhere in the world addresses the many task-dependent aspects of timbre representation and relates them to auditory perception in general and music perception in particular. This program will extend work on scalar acoustic parameters that correlate with the perceptual dimensions of timbre to the psychophysics of time-varying parameters and higher-dimensional representations such as spectrotemporal receptive fields modeled on auditory cortical neurons. I will confront these different models of sensory representation and feature extraction to assess their ability to account for perceptual results related to similarity perception, perceptual fusion, auditory stream formation, auditory salience, recognition of sound sources, and memory effects. I will determine how perceptual representations capture the way timbre, pitch, and playing effort covary in instrument sounds and will study their contribution to recognition and identification through the acquisition of mental models of sound sources. I will define how the relations between auditory representations of concurrent sounds condition their perceptual fusion and determine the resulting blended timbral qualities. I will study the aspects of timbre representation that affect auditory stream segregation and that capture listeners' attention by intrinsic salient timbral properties or extrinsic contextual salience. Finally, I will investigate the ways in which timbre is encoded in auditory memory and will study the impact of memory traces and perceptual models of sound sources on timbre dissimilarity structures. This research program will take our understanding of timbre processing to the next level by introducing a quantitative framework involving time-varying properties, by systematically evaluating and comparing representation models in terms of accuracy, generalizability and parsimony on a wide range of data sets, by characterizing the malleability of feature representations under different perceptual tasks, and by demonstrating that timbre contributes to attention, learning and memory. The program will provide a unique training environment for the acquisition of skills in behavioural testing techniques, experimental design, complex data analysis and statistics, mathematical modeling of auditory processes, and digital signal processing. It will have an impact on a variety of application domains such as auditory prostheses, music recommendation, and computer-aided orchestration.**