Spatial Analysis of Audio Images for Computational Musicology

音乐学家追求音乐的学术分析与各种目标，包括探索和理解创造性的音乐过程的愿望。 不断发展的计算音乐学领域带来了这种追求的计算方法。 拟议的研究计划将围绕当代现场电子音乐中具有空间声音特征的创造性音乐过程的调查开发计算方法（即，呈现为使得听众可以感知声源的方向和位置，通常利用环绕扬声器系统来完成）作为音乐的一部分。 它建立在与音乐学家同事（由SSHRC资助）和NSERC-DG资助的图像和信号处理以及声音合成工作的基础上。 对空间声音的强调意味着工作将集中在声场数据上-在我们的情况下，由高阶立体混响麦克风捕获（即，具有分布在球体上的大量麦克风元件的球形阵列）。 为了说明，考虑Clarke等人的工作。（2013）-他们通过手动识别图像中与突出音乐对象相对应的部分来分析音乐表演片段的频谱图（显示频率和时间分解的声音的图像），最终构建音乐的符号表示。建议的研究遵循克拉克的范式，但有以下重要的区别。 首先，我们将带来我们在计算机视觉和图像/信号处理方面的经验，以自动识别突出的音乐对象。 显着性的概念并不是微不足道的-我将以主题的音乐学知识和计算机视觉领域的进展为指导。 其次，我们将专注于空间声音，构建和依赖麦克风阵列和高阶高保真度立体声录音。 这使我们能够遵循O 'Donovan等人（2007）的领导，将麦克风用作音频摄像机，从而可以将球体上的二维音频数据处理为图像。 最后，我们将利用我们不断增长的现场空间电子音乐录音数据库。 我们现在拥有近80 GiB的当代空间电子音乐的高阶立体混响录音，这构成了这类研究的独特资源。 虽然研究的重点是计算音乐学，但该研究具有更广泛的应用。HQP将在任何需要声场分析的新兴领域工作。 两个重要的应用领域是环境噪声评估和改善，以及虚拟现实（VR）系统。一家本地的音响工程公司已经与我们合作，在工业噪音的测量和分析方面进行合作。伴随VR图形的声音系统还处于起步阶段。HQP在声场方面的专业知识在VR的未来将是必不可少的，我的HQP将有与VR文化应用专家合作的机会。

Musicologists pursue the scholarly analysis of music with a variety of goals, including the desire to explore and understand creative musical processes. The evolving field of computational musicology brings computational methods to this pursuit. The proposed research program will develop computational methods around the investigation of creative musical processes in contemporary live electronic music that features spatial sound (i.e., rendered so that the listener can perceive the direction and location of a sound source, normally done with a surround speaker system) as part of the music. It builds on prior work done with musicologist colleagues (funded by SSHRC) and NSERC-DG-funded work on image and signal processing, and sound synthesis. The emphasis on spatial sound means that work will focus on sound-field data - in our case captured by a high-order ambisonic microphone (i.e., spherical array with a large number of microphone elements distributed over the sphere). To illustrate, consider the work of Clarke et al. (2013) - they analyze a spectrogram (an image showing sound decomposed in frequency and time) of a fragment of a musical performance by manually identifying parts of the image that correspond to salient musical objects, ultimately building a symbolic representation of the music. The proposed research follows Clarke's paradigm, but with the following important differences. First, we will bring our experience in computer vision and image/signal processing to automate the identification of salient musical objects. The notion of saliency is not trivial - I will be guided by both musicological knowledge of the subject matter and progress in the field of computer vision. Second, we will focus on spatial sound, building and relying on a microphone array and high-order ambisonic recordings. This allows us to follow the lead of O'Donovan et al. (2007) in using the microphone as an audio camera, making it possible to process two-dimensional audio data on a sphere as an image. Finally, we will be exploiting our growing database of live-spatial-electronic music recordings. We now have close to 80 GiB of high-order ambisonic recordings of contemporary spatial electronic music, which constitutes a unique resource in this type of research. While the focus of the research is computational musicology, the research has broader applications. HQP will be well equipped to work in any of the emerging fields that need sound field analysis. Two important application areas are environmental noise assessment and amelioration, and virtual reality (VR) systems. A local sound engineering firm is already engaging with us to collaborate on measurement and analysis of industrial noise. Sound systems that will accompany VR graphics are in their infancy. HQP expertise in sound fields will be essential in the future of VR, and my HQP will have the added advantage of collaboration opportunities with experts in cultural applications of VR.