Spatial Analysis of Audio Images for Computational Musicology

音乐学家追求对音乐进行学术分析的目标多种多样，包括探索和理解创造性音乐过程的愿望。不断发展的计算音乐学领域为这一追求带来了计算方法。拟议的研究计划将围绕对当代现场电子音乐中创造性音乐过程的调查开发计算方法，这种音乐过程以空间声音为特征(即，呈现时使听众能够感知声源的方向和位置，通常使用环绕扬声器系统进行)。它建立在之前与音乐学家同事所做的工作(由SSHRC资助)和NSERC-DG资助的图像和信号处理以及声音合成工作的基础上。对空间声音的重视意味着工作将集中在声场数据上--在我们的例子中，是由高阶双声麦克风(即球面上分布着大量麦克风单元的球形阵列)捕获的。*为了说明，请考虑Clarke等人的工作。(2013)-他们通过手动识别图像中与突出音乐对象相对应的部分来分析音乐表演片段的语谱图(显示在频率和时间上分解的声音的图像)，最终构建音乐的符号表示。这项拟议的研究遵循克拉克的范式，但有以下重要区别。首先，我们将利用我们在计算机视觉和图像/信号处理方面的经验，自动识别突出的音乐对象。显着性的概念不是微不足道的--我将以主题的音乐学知识和计算机视觉领域的进展为指导。第二，我们将专注于空间音响，建设和依托麦克风阵列和高阶两性录音。这使我们能够追随O‘Donovan等人的脚步。(2007)将麦克风用作音频摄像机，使得能够将球面上的二维音频数据作为图像进行处理。最后，我们将利用我们不断增长的现场空间电子音乐录音数据库。我们现在拥有近80 GiB的当代空间电子音乐的高阶两性录音，这构成了这类研究的独特资源。*虽然研究的重点是计算音乐学，但该研究有更广泛的应用。HQP将具备在任何需要声场分析的新兴领域工作的良好条件。两个重要的应用领域是环境噪声评估和改善，以及虚拟现实(VR)系统。一家本地的音响工程公司已经与我们接洽，就工业噪音的测量和分析进行合作。与VR图形配套的音响系统还处于初级阶段。HQP在声音领域的专业知识在VR的未来将是必不可少的，我的HQP将拥有与VR文化应用方面的专家合作的额外机会。**

Musicologists pursue the scholarly analysis of music with a variety of goals, including the desire to explore and understand creative musical processes. The evolving field of computational musicology brings computational methods to this pursuit. The proposed research program will develop computational methods around the investigation of creative musical processes in contemporary live electronic music that features spatial sound (i.e., rendered so that the listener can perceive the direction and location of a sound source, normally done with a surround speaker system) as part of the music. It builds on prior work done with musicologist colleagues (funded by SSHRC) and NSERC-DG-funded work on image and signal processing, and sound synthesis. The emphasis on spatial sound means that work will focus on sound-field data - in our case captured by a high-order ambisonic microphone (i.e., spherical array with a large number of microphone elements distributed over the sphere).******To illustrate, consider the work of Clarke et al. (2013) - they analyze a spectrogram (an image showing sound decomposed in frequency and time) of a fragment of a musical performance by manually identifying parts of the image that correspond to salient musical objects, ultimately building a symbolic representation of the music. The proposed research follows Clarke's paradigm, but with the following important differences. First, we will bring our experience in computer vision and image/signal processing to automate the identification of salient musical objects. The notion of saliency is not trivial - I will be guided by both musicological knowledge of the subject matter and progress in the field of computer vision. Second, we will focus on spatial sound, building and relying on a microphone array and high-order ambisonic recordings. This allows us to follow the lead of O'Donovan et al. (2007) in using the microphone as an audio camera, making it possible to process two-dimensional audio data on a sphere as an image. Finally, we will be exploiting our growing database of live-spatial-electronic music recordings. We now have close to 80 GiB of high-order ambisonic recordings of contemporary spatial electronic music, which constitutes a unique resource in this type of research.******While the focus of the research is computational musicology, the research has broader applications. HQP will be well equipped to work in any of the emerging fields that need sound field analysis. Two important application areas are environmental noise assessment and amelioration, and virtual reality (VR) systems. A local sound engineering firm is already engaging with us to collaborate on measurement and analysis of industrial noise. Sound systems that will accompany VR graphics are in their infancy. HQP expertise in sound fields will be essential in the future of VR, and my HQP will have the added advantage of collaboration opportunities with experts in cultural applications of VR.**