Spatial Analysis of Audio Images for Computational Musicology

计算音乐学中音频图像的空间分析

• 批准号: RGPIN-2019-03974
• 负责人: Boyd, Jeffrey
• 金额: $ 1.68万
• 依托单位: University of Calgary
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=714173
• 关键词: Spatial; Analysis; Audio; Images; Computational

Musicologists pursue the scholarly analysis of music with a variety of goals, including the desire to explore and understand creative musical processes. The evolving field of computational musicology brings computational methods to this pursuit. The proposed research program will develop computational methods around the investigation of creative musical processes in contemporary live electronic music that features spatial sound (i.e., rendered so that the listener can perceive the direction and location of a sound source, normally done with a surround speaker system) as part of the music. It builds on prior work done with musicologist colleagues (funded by SSHRC) and NSERC-DG-funded work on image and signal processing, and sound synthesis. The emphasis on spatial sound means that work will focus on sound-field data - in our case captured by a high-order ambisonic microphone (i.e., spherical array with a large number of microphone elements distributed over the sphere). To illustrate, consider the work of Clarke et al. (2013) - they analyze a spectrogram (an image showing sound decomposed in frequency and time) of a fragment of a musical performance by manually identifying parts of the image that correspond to salient musical objects, ultimately building a symbolic representation of the music. The proposed research follows Clarke's paradigm, but with the following important differences. First, we will bring our experience in computer vision and image/signal processing to automate the identification of salient musical objects. The notion of saliency is not trivial - I will be guided by both musicological knowledge of the subject matter and progress in the field of computer vision. Second, we will focus on spatial sound, building and relying on a microphone array and high-order ambisonic recordings. This allows us to follow the lead of O'Donovan et al. (2007) in using the microphone as an audio camera, making it possible to process two-dimensional audio data on a sphere as an image. Finally, we will be exploiting our growing database of live-spatial-electronic music recordings. We now have close to 80 GiB of high-order ambisonic recordings of contemporary spatial electronic music, which constitutes a unique resource in this type of research. While the focus of the research is computational musicology, the research has broader applications. HQP will be well equipped to work in any of the emerging fields that need sound field analysis. Two important application areas are environmental noise assessment and amelioration, and virtual reality (VR) systems. A local sound engineering firm is already engaging with us to collaborate on measurement and analysis of industrial noise. Sound systems that will accompany VR graphics are in their infancy. HQP expertise in sound fields will be essential in the future of VR, and my HQP will have the added advantage of collaboration opportunities with experts in cultural applications of VR.

音乐学家追求音乐的学术分析与各种目标，包括探索和理解创造性的音乐过程的愿望。不断发展的计算音乐学领域为这一追求带来了计算方法。拟议的研究计划将围绕当代现场电子音乐创造性音乐过程的调查开发计算方法，这些音乐以空间声音为特征（即，呈现使听众能够感知声源的方向和位置，通常使用环绕扬声器系统完成），作为音乐的一部分。它建立在先前与音乐学家同事所做的工作（由SSHRC资助）和nserc - dg资助的图像和信号处理以及声音合成工作的基础上。强调空间声音意味着工作将集中于声场数据-在我们的案例中，由高阶双声麦克风捕获（即，在球体上分布有大量麦克风元件的球形阵列）。